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irSummary of Findings and l\nplicatibris 



The major findings of this Evaluation Report of 
Multiple- Funded Programs, 1976-77 are sum- 
5,marized below jn the order in which Jthe findings" 
are presented in Chapter V: enumeration data on 
prograip participants; background characteristics of 
prograrii* participants; ► indicators of ' program 
quality; and student achievement - results. A sum- 
mary of the findings from the special studies 
described in Chapter VI is also included. A full 
L report o( the findings from the special studies will 
be presented in a companioh publication entitled 
Report on Special Studies of Selected Schools 
Mith Increasing and Decreasing Reading Scores, 
1976-77, The implications of the findings are 
revie>yed briefly at the end of this chapter. 

( Enumeration Data on Participants 

Information gathered on program participants 
indicated the following: 

^ • More than 1,133,000 students in kindergarten 
through grade twelve were served through tlje 
combined funding sources. This figure repre- 
sents a^l percent increase sincfe 1975-76, and 
it represents 27 percent of the total enroll- 
ment in kindergarten through grade twelve 
^California's -pubUc schools. Of the program 
participants 72 percent were enrolled in 
kindergarten through grade three; 20 "ptTfiefit 
were in grades four through six; and 8 percent 
were in grades seven^ through twelve. TTiirteen 
percent of the participating students were 
limited- or non-English-speaking students. 

• More tjjan 1,000,000 students ' pceived 
' services in the rfeading, multicultural educa- 
* tion, language development, and/6r mathe- 
matics Qomponents. £ 

• More volunteers were involved in school 
.... programs than ever before. About 138,000 

adult volunteers and 90,000 cross-ale (stu- 
dent) volunteers worked in the schools, as 
compared to a total of 81,000 volunteers of 
I both types in' 1975-76. . 



ids, ESEA Title I funds, bilingual 
and/or Miller-Unruh funds tended 



. Backgroibid Characteristics of Participants 

Compared Ito* nonparticipants, students who 
participated iii school prograjns supported bj^earJy 
childhood education funds, educationally disadvan- 
taged youth 
education fund! 
Jo': 

• Hkve lowensocioeconomic backgrounds^ 

• Include a large proportion of students who 
spoke English plus a second language. 

Be more mopile/transient. 

Suci) significant differences in student background 
characteristics' make general comparisons between 
participants and nonparticipaats inappropi;iate. 



Indicate 
Two indicators 



^s of Program Quality 

'program quality -were utilized. 
(1) reviews of schiol-leveL plans; and (2) program 
quality reviews ccmducted during monitor and 
review <MAR) school visitations. The information 
gathered from the renews indicates the following: 

• More than 3,200 elementary school plans and 
460 secondary School plans were reviewed for 
compliance with applicable statutes and regu- 
lations. I ' 

• The ngionitor and review daia indicated that 
^ all sections of tne review instrument yielded 

arl ayerage of "good" ratings for all schools. 
Those grades wim ECE funding, either alone 
. or in combination wj^^h other sources, had 
consistently higher ratings than grades with 
' ^ EDY. and/or ESEA Title I funding alone. 

Student Kchiey^ment ^ 

District- reported ' tlata from standardized , 

nationally normed testsandicated tliat: 
1 

• Across jail grades students in multiple-funded 
schools typically i exceeded the national 
average by 1.2 standard score points in 
reading from the pretest to'the post-test. 



1 



' • Except in grade eleven the test score gairis in 
reading and mathematics of all participating 
fluent-English-speaking students exceeded the 
national average^ ' , 

• In grad^ three participating students scored 
slightly above the national average on the 

, reading post-test. 

• In grades one and two, students scored above 
the national arverage on the post-test in 
mathematics; students'in grade one exceeded 
the national average by 5.1 standard score 
points from the pretest to the post-test. 

• Students in ECE schools scored at or above 
the national average on the post-lest in, 
reading and progressed at a rate greater than 
the national average. 

• Students in ECE schools scored from 0.5 to 
^ 2.6 standard score points above the national 

average on the post-test in mathematics; 
^ students in grade one showed the greatest 
growth, exceeding the national average by 5.0 
standard score points from the pretest to the 
post-test. f 

• The average pretest score m readmg m'ECE- 
only schools was above the national average. 
On the post-test the stydents in these schools 
exceeded the national average Ijy 1.7 to 3.8 
standard score points. 

• Data of limited reliability were reported for 
limited- and non-English-speaking students. 
While limited-English-speaking-stiuj^ts parti- 
cipating' in consohdated programs .scored^ 
below the publishers' norms* on the pretest in 
both reading and mathematics, they tended to 
progress a.t a rate greater than that of the 
publishers' ^orm group, except in grade'ten. 

• Across grade levels a greater fluctuation was 
noterf in gain scores fof schools inr which 
testing was Idone on an annual basis than for 
schools in which testing was done on a^ 
within-Vear bj^^is. No explanation could be 

>^ffered for this finding. - 

California Assessment Program (CAP) data on 
reading achievement in glades two and three 
showed. the following: 

/ 

• Historical profiles of. reading achievement 
niuring the period 1973-74 to 1976-77 indi- 
cated that grade three students in ECE gener- 
ally had higher grad^^scores than the students, 
in those schools had before the schooLi 
entered ECE. Reading achievement in ECE 
Phase I schools and Phase IV schools in- 
creased slightly; reading achievement in Phase 
III schools decreased slightly; and achieve- 



mei)t in Phase II schodls increa^d sub 
stantially.*, 

• A longitudinal analysis oY California Assess- 
ment Program reading achievement scores 
from 1975-76 to 1976-77 sHow^ tjiat stu- 
dents in ECE schools made ^eater gai ns than 
students in a similar group of npn-ECE 
schoqls. * 

• Changes in average residual grade thr.iJe read 
ing achievement scores, on tne basis of the 
number of years a school had participated in 

.* ECE, showed improvement in a^;hi§/ement 
'with years of participation.^ Tf)e analysis 
indicated a small but' positive chang^'. 

Findings from the Special Studies 

A limited trend toward' declining tjiird grade 
residual reading scores in ECE schools whose 
* 1973-74 Entry Level Test scores were in the lowest 
20 pei-cent was reported in the EvaldSf^n Report 
of ECE. ESEA Title /. and EDY, 197^76] That 
finding leyi to two special follo\y-up stiidiei that 
were designed to examine in-d'epth the cHarakeris- 
«i;^tics of ECE schools with increasing and decreasing 
scores. A skidy of school processes by the Depart- 
ment of Ec^ucation Showed the following:^! 

• Although breakdowns in the translktioli* of 
the intent of ECE into educationalii exberi- 
ences for children were observed |n both 
types of schools Cthose with 'declining scbres 
and' those \^th increasing scores), th^y Vvere 
piore evident in the schools with de(:;re^iing 
scores. ' • 1 T 

• Positive leadership and management of 
change", at both the district and schoollevils, 
characterized the * schools with increasing 
scores; a lack of leadership or ineffective 
leadership was noted at the.schoolsl wijth 

' decreasing"Scores. \ 

• Teachers' attitudes and expectatipns ' wete 
significant factors. Where teachersi |were 
accou ntable for their students' perforitian^e 
and received accurate it^formation aboift th 



Phase I schools entered ECE in 1973-74, Phase II schools 
entered in 1974-75, Phase III schQols entered in 1975-76, aiJd Phas 
IV ^hools entered in 1976-77. 



A residual scure is the difference between the predifTed 
score of the students in a school and the actual score. *^ 



average 



Generalizations about the findings should~"Be^onfmed 
study population of ECE schools tjiat stored in the lovtest 
percent of schools on the 1973-74 Entry LevelTest (grade o: 
that had an increase or decline in Cahfornia Assessment 
thud grade reading achievement residual scores from 1973 U 
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performance, their behavjor was likely to have ' 
a positive effect on students' test scores. 

• Staff development activities that were closely 
tied to the instructional program \yere found*/' 

\to be' more effective than those of a non- * 
specific iDatur?. Courses, in multicultural 
understanding had a positive effect on .the 
achievement of limited- and non-English- ' 
speaking^ students. ' . , , ^ 

• Attempts to individualize the instryctional 
program with the advent of ECE had positive 

' effects, on student achievement when people 
were responsible for learning; negative effects 

Vera noted in tliose schools in which the ^ 
emphasis was on programs and materials. 

• Reading scores declined in schools in which 
^ the curriculum consisted primarily of reading; 

scores increased in schools in which students 
, had an opportunity to apply reading skills in a 
raijge pf curricular areas. 

• Four problems with regard to evaluation were 
common in all schools regardless of whether , 
their scores .were increasijig or decreasing; (1) 
school personnel had minimal information or 
misinformation about the tests and test 

, results, (2) school personnel failed to use the 
information that was available; (3) because 
monitor an3 .review ratings' and test scores 
measure different aspects of school func- 
tioning, schools sometimes had difficulty inte- 
gr^ing apd interpreting both' types of infor- 
mation; and (4) the meails for identifying arf3^ 
assessing the -progress of limited-English- 
, , speaking and /non-English-speaking children 
and for '^evaluating bilingual education pro- 
grams were inadequate or nonexistent. 

A special study of^classroo^ processes waT 
conducted by SRI International. Half of the 
schools in the study were also part of the Depart- 
ment's study sample. ^ " • 

An^^analysis of the relatiorvship between reading 
achievement scores and classroom instructional 
process variables showed the following: 

• A positive relationship was found between a ^ 
high third grade reading achievenrient gain and 

a low ratio of students to adults. 

• Students in classrooms in which the ratio of 
students to adults was low were absent less 
often than t^ose in classrooms in whicR the 
ratio was high. 

• Highly controlled classroom environments in 
which teachers provided systematic instruc- , 
tion and a great deal of positive reinforcement 



contributed to gains in third grade reading 
achieverH^ent. 
^ Smalf-group instruction was more efficient 
than work done with one child at a time 
during the class period. Jn classrooms in 
which reading achievement gains were small, 
adults often worked .with one child' at a time, 
and the other children did not receive the 
supervision and guidance that they needed to 
continue'to wofk on a task. ' 

• Classrooms within schools tended to be very 
differertt in terms df both the achievement 
gains made by children .and the instruct^5nal 
processes used by the teachers. • \ 

Major Imphcations from the Special Study ^ 
of Selected ECE Schools 

Th^ imjJhcations of the above findings were as 
follows: ' 

The expectations for what children can learn 
should be raised in many schools, especially 
/ " those with high minority populations. This 
includes the * teachers' expectations with 
regard to: 

1 . The' types of activities for children (reading 
and writing words, sentences, and par^^* 
graphs rather than" just fiUing^in the blanks, 
providing rote responses, -playing non- 
instructional games, working puzzles, and 
working with expensive media materials 
and equipment) 

2. Th^pace., of each child's , progress (each 
chii^ making optimal progress consistent 
with the Child's development rather than 
each child working as slowly as he or she 
desires to work) ^ 

j| 3. The quality of children's work (regardless 
^ of the type or level of work, good work- 
manship rather than half-hearted, sloppy 
•efforts) 

. • Communifcation-whether wVitten or or^l, 
from 'the state, from the district, or from 
within the school— should be more frank and 
personal, and the emphasis should be on the 
primary i^ole of human beings in the educa- 
tional processes. Too much pf the infor- 
mation received— by^eachers, parents, and 
children is couched in the abstract language of* 
"programs," which does not help people 
understand what they are to do. 

• The importance of good leadership, regardless 
of its source (principal, reading coordinator, 
group of teachers, or district) is fundamental. 
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Good leadership er^tails, among other things, 
close contact and ifivolvement with those who 
have operational Tesponsibility for* imple- 

^ . jn'enting decisions, such as teachers and aideS;_" 

mutual mist and reapect; anticipatory plan- 
ning, anowsopport for ongoing skills develop- 
'Voients. These and other good leadership 
Characteristics- should be' fostered through 
personnel practices and policies, especially the < 
" appointment df school principals, and 
through staff development programs specifi- 
cally designed to improve ^the Jeadership 
capabilities of those in leadership roles at the 
school, district, and state levels. 

• Staff development programs should foster a 
plear ^ense of purpose and commitment to the ' 
program and should help personnel integrate 

. . new programs and skills with existing prac- 
tices. The emphasis in staff development 
, should be on helping people to do their jobs 
. better: Staff development programs should be 
people specific; that is, aides should be 
grained -for the aide's job, and teachers should 
be trained for the teacher's job. ^ 
> • Teachers should pay close and frequent atten- . 
tion to how each 9hild responds to instruction 
. and should occasionally use informal assess- 
meht devices to check student progress. Initial . 
instruction should be provided by the teadher 

* (or some other person) rather than^by means 
of media, materials, 6r learning, stations. 

* Studen|5 should receive instruction in a 
variety * of ^ reading, skills and should have 

^opportunities to use and apply the skills in 
various curricular ^reas. The curriculum 
should be suc^ thqt the needs^of all students 
^ can bexnet, and especially those of limited- 
arid non-*Ei^glish-speaking studeYits'. 

• In ^ome^chopls with full bilingual education 
programs, heavy emphasis is pjaced on in-* 
5tructi^ in^the students' primary language . 
during thS^rlearly years of Elementary school; 
in the later years emphasis is ^ placed on- 
instruction in English. A need exists fqr some 
way of assessing the students' proffess during 
bo^h phases of their elementary school educa- 
tion. • ' ' ' ' ' 

The specific implications for Department of ^ 
Education action are discu§3ed beloy. The current^ 
practice of relying heavily on numerical indicators 
of student achievement and program quality v 
ratings provides information that is toa limited to 
provide for adequate exammation of the impact of 
funded programs at the school Ifevel. The depart- 
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ment is initiating •action in six areas to build a 

broader, mofe useful information base for program 

evaluation and program improvement: 
* * 

■-^#-Ill^-c~bo~p^faition with local educationaT 
agencies, the Department is developing in 
1977-78 ineans to use . other indicators of the 
impact of programs, including indicatprs of 
affective development, student attitudes, and 
schobl environment, 
•,^-The-vinany practical findings of the s'pedal 
^tudies indicated that the approaches in- ' 
^volving the case study and classroom process 
observations are effective in examining the 
nature and impact of programs. Such 
approaches will be utilized -again. 

• The Department will assist in local evaluation 
efforts by developing a network among school 
districts for the exchange of information 
regarding special evaluation and research 
studies. The network will facihtate"^ the 

' exchange of information- both,.j||^dy tech- 
niques and findings-among people witli/<^om- 
mon interests and will include newsletters, 
network exchange services, and technical 
assistance.* , ^ 

• Current methpds of reviewing school pro- 
grams are being reexamined in light of experi- 
ence as well as the provisions of AB 65. This 
reexamination .will include consideration .of 

^ various alternative approaches to program 
•review, while ^m^ntaibiing the consistenpy of 
ratings that is ii^cSSsary across schools. -Alter- ' 
natives that mayNbe considered include in- 
depth reviews of the^special needs of schools 
in which student achievement is low and 
. review processes that focus on tlije unique 
characteristics of intermediate .anS secondary * 
schools. ^ ' " • . 

Because reasons for^ year-to-year changes in 
school residual scores (fe^^gd on California 
. M Assessment Program Reading T^st data) are 
not clear, the Department plans to improve 
the capacity of Department personnel Jo 
understand and efxamine school situations 
that may influence performance on the Cali- 
fornia Assessment Program Reading Test 
Field service consultants, who regularly pro-* 
vide technical ^ssi;5tance and conduct program 
reviews at' schools, will receive training from 
California Assessment Program personnel in 
the interpretation •of aggregate and school- , 
level data. This training will be designed to 
assist the field service perspnnel in their 
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technical assistance 
efforts. ^ 



and 



program review 



• Since a surpnsLngly large fluctuation in gain 
scores . was 4iot^d i^oss- grade ^ levels, the 
Department will conduct a thorough analysi*s 
of the relative advantages , of annual testing as 
compared, to within-year testing. 

^ The J)robIem of assessing the progress of limited- 
and non-BnglisK-speaking students Js a concern of 
both schoolj districts and the Department the 
problem is not just the lack of appropriate testing ^ 
instruments, an even^ more pressing need exists to 
define better the desired nature and types of 
services for'limrted- and non-English-speaking stu- 
dents at the school and classroom* levels. Schools 
are statutorily required to provide fnstfuc^ion in a 
language that is understandable to students who . 
have/ been identified as limited- or non-English 
speaking, but instructic^nal goals and desired out* • 
comes, have not yet been .developed for such' 
students, ^t is essential fo discover what instruc- 
tional practices are "thost effective in meeting the 

^current md long-'term ''academic and linguistic 
needs of' limited? and non-fen|flisE-speSkin^ st*^ 
dents. Toward addressing ti^BLconcerns, tbe 



Department has formulated the following plans 
and/ or taken the following action?: ' 

• To Rii)vide for a cofierent instructional pro- 
gramjjthe "DepaTtifient'has^developed guide- 

• lines for interpredng the various requirements 
of ESEA Title Vli; AB 2284, and AB 1329. 

• .To provide, for greater specificity in the 

bilingual education program , prototypes 
described in the enabling legislation, Yhe 
Department is preparing operational defini- 
tions for dissemination to schools and ^chqol 
districts. 

• To determine what types of curricula arid 
instructional practices are most effective for_ 
fimited- and non-English-speaking stujdenti, ' 
^he Department including an ex>tensiv'e case 

. study review of selected "biiiirgual education 
programs in its evaluation plan. 

• To make known the bilingual assessment 
instruments tl\at do exist, tlie Department ha? 
^Jeveloped V*n* annotated bibliograpliy of 
selected acltievem^nt, affective, behavioral, 
diagnQ|fic, and language proficiency instru- 
mepl^ The Legislature* has -also, provided 
funds for the development of an'iachievemen't 

. test in Spanish. • . 

\^ ■ ' ' ' - - 



IL Introduction 



This Evaluation Report'ofMukiple-Funded Pro- 
grams, 1976-77 is designed to provide a description 
and interpretation of the effects in' 1976-77^ o'f 
local school programs funded <vith early childhood 
education* (ECE) funds, ^educationally disadvan- 
taged youth (EDY)iunds,' "funds provided under 
provisions of Title ^ L of the Elementary and 
Secondary Education Act of 1*965 (ESEA Title I), 
bihngual education (AB 2284> funds, and Miller- 
Unruh Basic Reading Act funds. Although these 
funds are* allocated through separate statutor> 
authorizations, their administration has been con- 
solidated td improve school projgrams through a 
comprehensive process of systematic program'^n- 
ning, implementation, and evaluation at the district 
and school levels. This comprehensive approach to 
program unprovement reflect^, the cooperative, 
-efforts of the Legislature, the State Board ,of 
Education, and the Departrrteyit of Education to 
support district and school efforts' to provide 
quality educational^ programs for all students in 
California. Tbese efforts incotporate three dimen- 
sions: (1) the rpovement to\4^d an equitable'ba^e. 
level of school support that tokens the disparities 
in wealth' impng districts; (2) the recognition that 
various levels of support and assistance are required 
to meet the specia^ needs of such students as file 
educationally disadvantaged, limited- and non- 
English speaking, and physically and mentally 
exceptional; and (3) the provision of support and 
encouragement to ensure . that funds and program 
improvement efforts to meet the unique educa- 
tional needs of all, students are coordinated 
through -a planning, implementation, and evalu<^ 
tion process at£^i school site. 

A variety ^paucational sery;^ecs^ .provided for 
by the additional funds from these^^urces'The 
funding sources represented in the schbols vary in 
accordance, witH the needs of the students. 
Approximately two-thirds o.f the public schools in 
California receive funds, from one or more of the - 
five funding sources examined Jn' this report. The 



con^bined annual expenditures froin these sources 
total approximately S^60 million. 

The responsibilities of the Department of Educa- 
tion for the evaluation of the effects ot programs 
funded by these sources are broad and complex. In 
addition to pubjishing this mandafed stale-level 
evaluation .report, the Department assists cfistricts 
with logal program evaluations and performs local 
{Jro^am compliance reviews and program qualit> 
reviews forj)urposes of progr^ improvement. The 
DepartmenT also conducts special follow-up studies 
of evaluation findings reported in previous >ears. 

Senate Bill 1698 (Chapter 791, Statutes of 
1976) amended Education Code Section 491.3 
(Section 33403 of the reorganized and revised 
code) to provide for a consolidatedl evaluation of 
programs funded under several different funding 
sources, including the following: ^ 

K-Early Childhood Education (ECE) Act 

2. Educationally Disadvantaged Youth (EDY) 
Act ^ ' " 

3. Elementary and Secondary Education Act 
(ESEA), Title I 

4. Bilingual Education Act of 1972 (AB 2284) 

5. Miiler-Unruh Basic Reading Act 

' In response to SB 1698 of 1976, the Depart- 
ment developed a comprehensive state-level evalua- 
tion deigned to examine ^program coordination 
resulting frofm the consolidation effort and to 
collect the required enumeration, implementation, 
and student achievement data. 

The evaluation was designed to address the 
following bro^d questions:/ 

i . For wjhat statewide educational purposes were 
consolidated application funds used? 



The Education Code was reorganized aigd revised in 1976. in 
subsequent citations fust refereij^s are the section numbers from 
tlie reorgaiuzed code (Assembly Dill 3100, Chapter 1010, Statutes 
of 1976, as amended by Assembly Bill 311)1, Oaffter 101 i. Statutes 
of 1*976), which became effective on April 30, 1977. Section 
numbers in parentheses or br^ckej$. are from the 1973 code, as 
amended. 
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2 What was. the extent of participation in " 
programs funded through the five sources? 

3 How much money was allocated through the 
various funding sources, and how was it 
spent? ' ' ' 

4. What was the nature of the implementation of* 
funded programs? 

5. What inferences could be made about, the 
quality of the school programs and services 
provided for bv th^ funding sources? 

6. What yiferences could be made abouf the 
performance of students who received servues 
in mMunded programs? 

• What were some characteristics of local 
' efforts'to evaluate schoo^ programs? 

In addressing these questions and compiling this 
repilrt. the Department used information from 
annual school and district reports, reviews of 
school plans, and on-site program reviews, student 
' achievement data, including California Assessment 




Program readmg scores and scores from pubhshers* 
norm-referenced te^ts, uiformation from a special 
case stud> of a selected sample of schools with low- 
achieving students, and information from a sample 
of sch6ol and* district evaluations. 

This report consists of seven chapters. Chapter I 
contains a summary of the evaluation findings and 
a summary of the implications of those fmdings. 
Chapter 11 'is the introduction to the report. 
Chapter IIF provides a description of the funding 
sources and the programs supported by Jlie funds 
from those sources. Chapter IV includes descnp- 
tions of the methodologies and procedures used in 
the evaluation and a discussion^TDf their advantages 
and limitations. The findings^ inchiding enumera- 
tion data, data on participants, and student 
achievement data, are presented in Chapter V. 
Chapter VI contains findings from the special 
studies of selected ECE schools v^th low student 
achievement. Characteristics of local evaluation 
efforts-are reported m Chapter VII. 

^ 

4 
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IIL Description of Funding Sources 
and School Programs 



Chapter III of the evaluation report contains 
descriptions of the funding sources, legislative 
intents, goals, specific eligibility criteria, and the 
scope and nature of the ^ district and school 
programs supported by the various funding 
sources. 

Firnding Sources 

.Five major funding sources are coordinated 
through the consolidated application and evalua- 
tion processes. Four are state sources. (1) Eariy 
Childhood Education (ECE) Ad, (2) Educationally 
Disadvantaiged Youth .(EDY) Act, (3) Bihngual 
.Education Act* (AB 2284), and (4) Miller-Unruh 
Basic Reading Act. The fifth fxinding source, the 
Eleinentary and Secondary Education Act (ESEA), 
Title 1, is a federal source. Participating schools 
may receive funds from as many as five or as few as 
one of the funding sources. They use the funds for 
which they are eligible in a manner set forth m a 
coordinated, comprehensive plan that is designed 
in^ accordance with the funding source program 
goals, specific student eligibility critenS^ and legis- 
lative intents. In their planning efTort schools assess 
school and individual student needs to define 
school program goals and to develop the necessary 
instructional J^d support services to achieve the 
goals. 

Chart III-l shows iTow funds, flow from the 
sources to provide for services to students. From 
the figure one can readily see that school and 
student eligibility for funds and services varies 
from source to. source. However, once the funds 
are a)iocated, they are integrated at the school level 
to support a variety of services designed to meet 
the umque needs of eligible students. Thus, in 
school receiving funds from more than one source, 
the separate sources are utilized to provide coordi- 
nated services, which, other than being targeted at 
eligible students, need not be identified indi- 
vidually on the basis of source of funding/ 
. The descriptions presented below ieflect the 
statutes and regulations that were in effect and the 



proceduFes that ^ere used during the 1976-77 
school year. The reader should note that recently 
enacted legislation, notably Assembly Bill 65 
(Chapter 894, Statutes of 1977), will markedly 
change the funding allocation procedures for 
various state funding sources. 

Early Childhood Education Act 

The early childhood education legislation 
. (Chapter 1 147, Statutes of 1^2) was designed to 
provide an '*^umbrelfa'' or framework under which 
primary education (kindergarten through grade 
three) in California could be reformed or restruc- 
tured. Integral to the umbrella approach is the 
coordination of categorical funding to provide 
additional services to meet the needs of special 
populations within a school. The primary goal of 
ECE is to provide for all students in kindergarten 
through grade three an education that is design^ 
to meet their unique needs, talents, interests, and 
abilities. The extensive participation of parents and 
the cdfnmunity in general in the edu<Jition o( 
children in the early grades was also called for in 
the ECE legislation. Early childhood education 
differs from other funding sources in that through 
ECE, funds are allocated ^to districts for the 
purposes of reforming and restructuring total 
school programs so that they can address the 
individual needs of all children in kindergarten 
through grade three^ The other funding sources, 
though allocated to districts for eligi&le schools 
and identified students, provide for supplemental 
seivices to subsets Of a schoofs population, not the 
entire kmdergarten through grade thipe popula- 
tion. 

Each year at least half of the ECE funds for an^ 
participating district .must be used at those schools 
that have the greatest "educational need.'* The 
measure of educational need is the percent or 
number of students scoring at or below the 25tii 
percentile on standardized, nationaUy normed 
reading or mathematics achievement tests, The 
other half of the ECE funds may be used. for any 
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CHART IIM s . K 
^ Descriptive Data on Funding Soarces for Multiple-Fundea School Programs, 1 976-77 , , 



Programs 



Data 
categories 


Early childhood 
education 


Educationally 
disadvantaged 
youth 


\ ESEA Tide 1 


Bilingual 
education 


' MiUer-Unruh* 


Eligible grades 


(K-3) 


(K«12) 


(K-12) 


CK-12) 


(K-6) 


Authorizing agency 
and allocation 
procedure 


Legislature; money 
to Department 
of Education 


Legislature, mone]^ 
tO'Department 
of Education 


Congress, money 
to U.S. Office of 
Education and 
U.S. Office to 
Department of 
Education; money to 
counties on ha^i^ of 
poverty index 


Legislature,money 
to Department 
of Education, 


Legislature, money 
to Department 
of Education' 


Basis on which 
Slate E)epartmenu - 
of Education dis- | 
tnbutes money 
to districts 


Success of EbE 
sctiools ^ / 

\ 


Index of low in- 
come, tran- 
siency, and 
iinuted- and non- 
English-speaking 
students 


Aid to Families 
with Djependent 
Children (AFDC) 
count 


Competitive proj- 
ect apphcation 


Previous participa- 
tion of schoob • 
within district 


Ka^ic nn whirh 

district distrib- 
utes money to 
schools 


^hnnf nian /'^O 
i>CIlUUl pidji 

peitent must be 
spent at schools 
with lowest 
achievement) 


Low achievement 

y : 


Poverty indexes 


School plan 


Need for reading 
specialist ^ 
position^ 


Basis on which . 
schools provide 
services 


To all K -3 students 
on basis of indi- 
vidual needs; use 
of other funding 
sources to aid \n 
program planning 
and coordination 
of services 


To identified 
K-12 students 
on basis of low 
achievement; 
priority given to 
yqunge^t students 


To identified 
. students 
on basis of low 
achievement; 
priority given' to 
youngest students 

^ • 


Bilingual services 
to classrooms 
identified in proj- 
ect apphcation ' 


To identified K-3 
> or 4-6 students 
oh basis of lo^ 
reading achieve- 
ment 
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conji)anation of schools tKat.tlie district selects, 
since all schools are eligible tor funding. ' ? , ' 

Up'on State Board pf Education approval of a ^ 
sthocjl's proposed program, ECE funds were alio- . 
cated to districts in 1976-77- cm the basis of SI 40 
per siudent enrolled in kindergarten through grade 
three. An additional S70 was aUocafed for each 
student who stored at or belo'lv the 25th percentile* 
of national norms in. reading or mathematics 
achievement. A district could not receive the 
additional 570 far more .than 25 percent of the 
participating students in the district. 

Educationally Disadvantaged Youth Act * ' 

State funding for educationally disadvantaged 
youth (EDY) was authorized by Chapter 1406, 
Statutes of 1W2 (SB 90). Educationally disadvan- 
taged youth funds are intended^ to provide funds to 
supp6rt quality educational services for low- 
achieving Students. 

• Educationally disadvantaged youth funds are 
allocated to school districts ih accordance with a 
formula that mdudes indexes of limited- and 
non-English-speaKing students, transiency, and 
overty . The Jformula is given in Appendrx A. 
Once district eligibility for EDY funding is 
estaBlished. districts select those school attendance 
areas that mclude the students -with the greatest 
educational need. School need is determinetl by 
either tTie number or percent of students scoring 
below the 25th percentile on standardized achieve- 
ment tests in readjng and mathehiatics. AU stu- 
dents who are in schools that receive EDY funds- 
and who score below the 50th ^ percentile" cm 
standardized achievement tests are eligible for 
services. Under provisions of the Edycationally 
Disadvan^ged Youth Act, EDY funds are used to 
serv^ only those students enrolled in public educa- 
tion pro-ams. 

Elementary and Secondary Education Act, Title I 

Title I pf the Elementary and Secondary ^duca- 
tipn Act of 1965 authorizes funding for edu- 
^tional programs that arfe designed to benefit 
students from low-income families. Title I funds 
lare allocated to Cahfomia on the basis of the 
number of children from low-income families in 
ea'ch county. The Department of Education then 
allocates funds to school districts within counties 
on the bas>$ of Aid to Families with Dependent 
Children (aFDC) data from the counties. Within a 
school distnct schools are ranked on. the basis of 
the poverty of the populations they serve. Schools 



with a povprty index exceeding the district average 
po\ffij^y Index are eligible for Title I funds. ♦ 

OncQ e|SEA Title I funds are allocated to a 
school, st^idents are selected for participation dn 
the basis j of their educational need; which is 
defineU to| include students scoring below the 50th 
percentile I on standardized achievement tests or 
those wl^p have serious learning deficiencies be- 
cause of linguistic, social, cultural, or economic 
isolation, ^ 

Whfle ESEA Title I funds may be expended Cor 
students at all grade levels through grade twelve, ^ 
state regulations require that the youngest eligible 
children within the district he* served first when • 
categorical aid monies are insufficient to serve all 
^^igible children. 

Students attending nonpublic schools are eligible 
for services prbvided for by ESEA Title I funds if 
they live in an eligible attendance area and are 
educationally deprived. ESEA Title I funds.are also 
/provided to^the state to serae handicapped students 
living in state mstitutions, migrant students, and 
students in state institutions for the neglected or 
delinquent. 

Bilingual Education Act of 197? 

'The Bilingual Education Act 'of 1972 (Chapter 
1258, Statutes 6f 1972) (AB 2^84) was enacted to 
establish bilingual, bicultural programs for limited- 
English-speak ing^ and non-English-speaking 
students. 

Districts submit project applications, -and funds 
are awarded through a competitive grant process. 
Programs are developed in - the context of the"^\/ 
intent of the Bilingual Education Act, which 
includes (f) the students' d^*^lopm^t-of fluency 
in English as well as compe^ce in their primary 
language; (2) the provision for positive reinforce- 
m^t of students' self^concepts; and (3) the devel- 
o^ent of intercultural awareness among students, 
parents, and fnstructioiial staff. 

« ^ 

Miller-Unruh Basic Reading Act ^ 

In 1965 the Legislature passed the >liller-Unruh 
Basic Reading Act (Education Code sections 
54100-54180 [5770 through 5798]) to prpvide 
reading specialists "for the prevention and correc- , 
tion of reading disabilities at the eariiest possible 
time in the educational career of the pupil. 
School eligibility for Miller-Unriih funds was deter- 
mined on the basis of reading scores at the time the 
funds were first allocated. The highest priority is 
given to students in grade one, second priority is 
given to students in grades t\yo and three. 
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♦ School Programs 

As was described in the previous section, funds 
from five majotr sources are appropriated for use in 
eligible districts and schools. School districts apply 
for funds from these sources by submitting to the r 
State Pepartment of Education Form A-127D, v 
Consolidated Ap^Hcation for Funds for Educa- 
tional Programs, On the basis of state regulations 
and district policy (for certain funding sources 
only), school districts determine the amounts from 
each funding source to be alli^ated to the schools 
in the district. 

School Restructuring Through Early Childhood Education 

Schools whose funding sources include early 
childhood education are, unique with respect to * 
schools with all other funding sources or combina- 
tion of funding sources. Because early childhood 
education f^nds are not ''categorical" funds, EC£ 
programs are designed for all kindergarten through 
grade three students at participating schools. The 
concepts of restructuring school processes and 
operations encompass the entire school in the 
primary grades. All funds received by the school, 
including basic aid, district support, and categorical 
funds, are to be used in support of ^ unified 
instructional program in which all of the restruc- 
turing concepts aiie incorporated. 

The instructional program in an ECE school- • 
^ should be such ' that the needs, interests, and 
'capabilities of eath child are provided for. The 
instructional program should also provide for 
instructional settings, groups, materials, and 
instruction that will enhance each child's academic, 
emotional, and social development; Each child in 
an ECE school should have access to assistance and 
attentiori from 'classroom teachers, aides, volun- 
teers, parents, and cross-age tiitore as needed. 
^ Ciirriculaj^ emphasis should be on rea3lrtg, mafhe- 
ijiatjcs, language development, and multicultural 
awareness. Children's health needs should receive 
appropriate attention. Parents ^ould be encour- 
aged to participate in classroom activities, as schpol 
^advisory committee members, and as volunteers in 
other ways; and they should have an opportunity ^ . 
to broaden their knowledge of their child's educ6-* 
tion. All kindergarten through grade three staff 
members should have opportunities to increase 
their knowledge and skills so as to be .better able to 
provide an effective instructional program. Infor- 
mation gathering, evaluation, and decision making 
are participatory; that is, tfie various members of 
the school community assist in making the deci- 
sions that affect them riiost. The needs of students 



with limited or no English language experipfrce 
should be addressed through instruction in English 
and in their primary language, and staff develop- 
ment programs designed to help the school staff 
better meet the needs of such students should be 
provided . ^ /^^^^ 

Students with special needs i/ECE schools that 
develop an overall kindergarten through grade, 
three school plan are to receive special Services in a 
manner that provides for coordination with the 
overall school plan. The special services are direc- 
ted toward only the studeifts with special needs, 
and tjiey are designed to 'serve as a valuable 
supplement to the services that all students receive. 
These services must be over and above those 
provided in the regdtar instructional program. 
Some examples of services over'and al^pve those of 
the regular program are additional time' and atten- 
tion froni aides, special assistance from a resource 
specialist 'teacher in the classroom, periodic visits 

' to a resource room or resource center that might 
have*' special audiovisual equipment, books, or 
instructional materials, additional time with the 
school nurse or. counselor, and "cultural enrichment 

, activities such as visits to museums, government 
buildings, parks, and other community facilities. 
All of the services just described could be provided ^ 
from the funds* allocated to , meet the students' 
special needs. ' " _ ' 

School Programs for Students with Special Nee^s. , 

Schools that receive funds to serve students with 
special needs are required to plan only with respect 
to the subpopulation of students with special 
needs. SUch schools include ECE schools that 
receive special funds to serve students in grades 
four through six. All areas of the curriculum are to 
be addressed in a plan that provides for instruction 
based on the identified needs of the low-achieving 
student$* only. Parent participation and parent 
educatior) are to be emphasized. All additional 
services are directed toward low-achieving students 
only. Staff development activities are required to 
enable teachers and paid a^d volunteer staff to 
provide the special services* for the identified 
students. Any of the special services described in 
the above section on ECE schools might be 
planned for in a school receiving funds only for 
students with spiecial needs. The program designed 
for students wi^ special needs shoyyld be well 
coordinated witlNthe school's basic instructional 
program. 

1. AIL schools that complete and submit a school 
level/plan must address within the plan (and^thus in 
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the school's' program) the special needs of the 
limited- and non-English-speaking studentr en- 
rolled. Such students are to receive instruction 
designed to improve their mastery of English, 
including their skills of speaking, reading, and 
writing. They are also to receive instruction in all 
curricular areas in their primary language until they 
are able to benefit from instruction in English 
only. Schools typically employ bilingual aides and 
bilingual resource teachers and purchase special 
materials in the second language to meet the needs 
of limited' and non-English-speaking students. 

While all schools must provide for the needs of 
limited- and% non-English -speaking students regard- 
less of the funding source(s) from which they may 
receive funds, some schools also have bilingual 
education programs funded under pro>isions of AB 
22,84 of 1972. The programs at those schools must 
include one or Inore classrooms in which a biUn- 
gual, biliteraje teacher provides instruction in the 
students' primary language and in English. HTwo- 
thirds of the students in a biUngual classroom must 
be limited- or non-English -speaking children, at 
least one-third must be fluent-English-speaking 
students. Student needs are to be assessed in both 
languages, and continuous assessment of student 
progress, instruction appropriate to the. individual 
student, and evaluation and modification of the 
instructional program should all be features of 
bilingual , education classrooms. Instructional 
materials and an environment that fosters under- 
standing and pride in the students' culture and 
language and in the dominant culture and language 
are to t)e provided. Staff inserviee training activi- 
ties, parent involvement activities, and other sup- 
port services should be provided in the same 
manner as in the programs previously described. 

Classroom aides in bilingual education class- 
rooms are to be bilingual, and students should have 
many opportunities to see, hear, and .speak their 
primary language. In addition to instruction in 
reading, writing, and other curricular subjects in 
their primary language, students should receive 
instruction in oral and written English as their 
mastefy of reading in their prirnary language 
Warrants. 

Secondary School Services for Students with 
Special Needs . 

"•Secondary schools are eligible only for funds to 
^be used for students with special needs (low- 
achieving students). The funds are to be used in the 
development and implementation of instructional 
services that i^re over and above those available to 



all students in the regular school program. Typical 
services are those provided by classroom aides, in 
special reading or mathematics laboratpries, and by 
teachers with special skills in remedial work. 
Classroom aides in secondary schools are trained to 
give assistance to those students in ttie class who 
are eligible for program participation. In some 
secondary schools special laboratories are provided 
through compensatory education (Title I/SB 90) 
funding. Students may be assigned to the labora- 
tory on a temporary basis as individ.ual assessment 
warrants such assignment. The laboratory experi- 
ence must supfJlement a student's regular class- 
room instruction. Specially skilled teachers may. 
assist identified students in the regular classroom 
or in a laboratory or resource center, using 
individual assessment procedures, educational 
media, and remedial materials provided with com- 
pensatory educatton funds. Staff development . 
activ^ies are reqtiired for all staff mepibers who 
have responsibiUties for the identified student 
participants. At the secondary level the^ supple- 
mentary instructional program is usually confined 
to reading and/or mathematics. Regulations require 
that the youngest students be served first, ajid 
,many programs in secondary schools serve students 
m the ninth and/or tenth gracies only since serving 
alf eligible students in these grades frequently 
e:<haustS the funds available to support the pro- 
gram. 

Schools with Miller-Unruh Reading Specialists 

Some schools with kindergarten through grade 
six have a reading specialist supported by Miller- 
Unruh Basic Reading Act funds. In schools that are 
involved in comprehensive school-level planning 
(and that receive funds from any of the other 
funding sources), the services of the reading 
specialist must be included in the school's instruc- 
tional plan as part of the integrated services that 
are availaj^le to meet the ne^ds o/ each student. 

Miller-Unruh reading teachers work .directly with 
students with reading deficiencies, providing 
remedial work and conducting other activities. 
Miller-Unruh teachers typically give first priority to 
first-grade students. The reading specialist may also 
work with classroom teachers as a resource person 
to help them. meet the varied needs of students in 
the area of reading. *No .standard Miller-Unruh 
program exists, rather, various patterns of services 
are provided by Miller-Unruh teachers. Services, 
whether in a school in which the M^iller-Unruh 
teacher is the only "extr^" or in a school in which 
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the instructional program is supported by several 
•funding sources, are determined at the school level. 

Summary • * * 

The ^ five consolidated applicati]^ funding 
^sources discusserfl^^n this report provide the neces- 
sary support -to implement schfiol' restructuring 
through E(rE and to providf to eligible students 
services abovVand beyond tt^bse available througli^ 
the regular instructional pro-am. 

School programs>developed through these fund- 
ing sources are of two types. ( 1 ) those designed to 
restrufture , the total program in kindergarten 
through grade three; and (2) those that provide 
services for studerits'with special educational needs. 

^ Restruc^turing efforts in ECE schools are charac- 
terized by (1) schoolwide planning for the restruc^ 



turine^rf the program in« kindergarten through 
grajjcthree to address the needs of all students in 
these grades; and (2) planning for the use and 
coordination of all available resources-basic aid, 
district support, and categorical resources within 
the overall "un>l?rella'' of the ECE restructuring 
plan. . \" . 

• Services for students with special educational 
needs are characterized by (1) targeting of services 
only to identified, eligible students; and (2) 
comprehensive planning] for - "over and above" 
services to ensure that they address the individual 
needs of each identified student. 

It is also important ito note here that since each 
participating school and its students have, unique 
characteristics, no single program or set of features 
characterizes a "model" program. 




IV. Methodology, Procedures,, and Limitations 



Chapter ' IV includes descriptions of the 
methods, procedures, and instrumentation used to 
compile and analyze the data for fhis evaluation 
report. The methods, described include, tKose that 
w^re used to collect d^ta to answer* questions 
about .program participation andvexpenditures, 
questions . related to the nature and the ^ality of 
schoo/ programs and services; and questions about 
the achievement of students who received services. 
The advantages and limitations of th^ major data 
sources and of the special technical studies are 
discussed. Also included is a. description of t^'e 
methodologies used to select schools for the specif 
case studies.^ 

Information and Data Collection Procedures 

« 

In examining the effects of programs funded* 
through the consolidated application funding 
source, the Department considered *the following: 

• Enumeration data, including allocation ancl 
^ expenditure information, describing the scope 

of' school programs supported through *the 
five funding sources 4 - ) 

• Indicators of school program quality ' 

• Indicators of student achievement in heading 
and mathematics 

Circumstances associated with increasing and 
decreasing reading scores^ 

• Examples of local evaluation efforts . 

, The reader should note that drawing inferences 
from a review of a single type o/ data, such as 
student acffievement data, would not provide fpr a 
.^'comprehensive look at the outcomes ofjMpgrahis. 
This report is therefore based upon Uperies of 
.dnaly^es of information pertaining to the areas 
described above. . ^ * . 

The squrces of data that' were xised in tlte^ 
evaluation of programs that received funds through 
the col\solidated application I'ri' i 976-77- are listed * 
in Chart JV-1. These data source^^.are described in 
terms of (l)|the level of the agency that completed 
the instrument, (2) the instrument that was used to 



gaJlier the data; (3) the agency that completed the 
instrument, aAd (4) the typ^ of data collected. 
Many pf the -instruments listed were also used fqr 
.other , purposes, such as deteripining participant 
eligibility arid planning at the scho(|^, district, and 
state levels^^. ^ ' . . 

Enumeration Dat^ on Participants * - ; 

All participating schools were asked to file an 
end-of-year i?eport that included information about 
the numbers of students who-.recei^d services 
provided for under each fuiftlng source, the 
numbers of volunteers, who worked in the pro-, 
grams, ^nci the numbers of personnel hired with 
funds frc^m the * various, funding sources. Self- 
reported enumefation data were verified by data- 
editing procedures established th^ Department. 

Allocation and Expenditure Data * ^ " 

. Informatibn about the percents of expenditures 
in the various budget categories, for each of the 
fiveJunding sources, was obtained by examining ^ 
sample of preliminary distric^fiijancial reports, 

ndicators of Program Quality 

As described earlier, a major goal' of the ECE 
restructuring effqrt is to improve instru^iori and 
thereby increase^ stodent achievement in basic 
skills. In most casefe suth improvement requires 
institutional changes *that ^fect the goals, roles, 
and environment of parScipatinp districts, schools, 
staffs, and commtmities. T^e collective effects of 
institutional chajjge are reflectecf in the quality of 
the {programs in iwtieip^ing .schools. 

In J 976-77 the Depamnent of Education u&d 
two methods to exaffline program quality. (1) 
reviews of school pr^am .plans; and (2) on-site 
progfdfti quality reviews aUd program c^mpiiahce 
reviews. >. * * . . 

School Plan Reviews . ' « 

All schools that received funds through the 
consolidated application vv|teVequiEed to engage in 
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" * ' > CHART I^-l • ■ ■ ' 

Data Sources Used in the Evaluation d^ECE, ESEATitle l.tlDY, Bilingui 
E4ucation, and ^iller-Unnih Programs, 1976-77 /| 



4. 



Level -of educational 
^ agencies 



Instruments) to 
gather data 



'Agency completing 
^iifetrument 



Tyjpes of datrf^ollected 



Enumerqfiofj^atg 

Elementary and secondary 
schools with multiple , 
funding 

Miller-Unruh 




Phase I, Consolidated 
Evaluation Report • 
(Form E-127P) 

VormJ?!^ % 



Yogram Quality 

Districts receiving multiple 
funding 

' Elementary a^d secondary 
schools with multiple 
funding 

Elementary schools with 
multiple funding ^ 

Selected elementary schools 



\ 



Form A-127D 



Selected sample of ECE ^ 
schools with /ieclining and 
incc^asing student 
achievement \ 

Student achievement data 

Elementary and secondary 
schools with multiple 
funding ' ^ 

All elementary schools 



Form A-127ES 
FormA-127Sec ' 

School-level plan critique 

Program Quality Review 

Instrument 
School Level Program 

Compliance Rwew 

Instrument 
District Level Program 

Compliance Instrument 

Special study 



lase I, Consolidated ^ 
Evaluation Report 
(Form E^127P> 

California Assessment 
'Program Entry Level Test 
"(grade one) and Riding 
Test (grades two ana 
three) \ 



Schools 



Districtoffice' 




State Department of. 
Education * 

State Department of 
Education * 



State Department of 
/^ucation and SRI 
International * 



Schools 



Schools 



Enumeration of pupils, program 
personnel, and volunteers 

Partidgating schools 



\ • 



trictJevel allocation plans and 
Application for funding ' 

School4evel pl^s> needs assessment 
report and description, objectives, 
actiyities, 6valuatiqn, dissemination 
and budget 

School^eJ pl^ui»sp6cifications / 

' ' * * 
On-site re^ew and rating of»program 
implementation \ 



School case studies and classroom 
observations ^ 



Pupil achieveihentx5n publishers* » 

,normrreferenced tests (pretest 

^'d post-te^) ' - > ' ^ - 

Assessment of entry-level educational 
skills at the beginning of grade 
one; reading^skiHs at the end'of 
grades two and three 



r 



a systenutii: planning process for ea||h of the 
following program components: 

• Langua|e'devclopment 

• Reading 

• Mathematics 

• Multicultural education 

• Staff development- 

• Parent participation 

• Parent dducatiqru ^ 
Health/auxiliary'Sfervices 

All elementary schools Jhat received ECE, EDY, 
f ,ESEA Title I, i>r bilingual education funds pre- 
' pared and submitted comprehensive program plans 
for I916'n. For the majority of the^ schools, the 
planning process was implemented ^s part of the ^ 
ECE restructuring efforts. Secondary schools that 
received EDY,. ESEA Title I, and/or -bilingual 
education funds conducted planning in much the 
same manner as the elementary schools; each 
submitted a school program plan/ 

School plans submitted to the Department of 
Education were reviewed by Department personnel 
• or by trained readers from offices of cfounty 
.supeyfintendents of schools or school district staffs. 
Two types of plan reviews were returned "^.to 
schools: (l) a compliance review, in which were • 
indicated the areas in which th^ school plan wash- 
out of compliance with regulations; and (2) a 
quality critique, in which possible weaknesses in 
the. program design were indicated. In previous 
, years quantifiable ratings were part of the quality 
critique,, but those ratings were discontinued, in 
1976-77 in an effort to reduce- the paperwork 
burden on the schools. 

On-iite complia(ice feviev^s and quality reviews 
of elerhintary school programs. Monitor and 
review visits were conducted to review school-level 
program compliance and to assess the quality of 
prggiram implementation. Monitor anid review 
teams of at least two persons visited schools during 
the period from October 1, 19\6,"to May 4, 1977. 
TTie teams spent tw^ days in each school that was 
receiving funds from more than one source and 



that ]iad an average daily attendance fa.d.a.) over 
1 75 and two days in each school that \yas reeeiviiig 
,^^^^^\y ECE funds and that had an a.d.a. ove|^250. 
Visits to other schools were for one day only. 

Monitor and review teams consisted of either a 
Department of Education- employee or other 
experienced individual, Vho acted as the lead 
person^ and a second team member, who was either 
an employee ofthe Department or other individual 
experieitced in the monitor ^d review procedure. 
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Personnel frpm offices^ of county superintfencients 
of schools and school districts comprised more 
than a majority of the monitor and reviejw leam 
members. . , ' ] \ 

Designing instruments and procedures jio assess 
program quality both within and across thousands 
of schools statewide is y difficult /asl/./the 
development of* measuring ^struments/ for y these 
purposes involves new assumptions and.niethads., 
When used within schools, the mopitor d^id review 
process is intended to providg-^s^Stematic feedback 
to a school about its program planning, imple- 
mentatidh, and ^valuation. This feedfeacki is 
designed to aid-in future program development by 
encouraging^ ongoing planning, internal mortitor 
ahd^review, and process and product evaluation. 
Across schools the monitor and review proc^ is 
designed to provide information regarding the 
quality of various aspects of s9hool programs. 

The t.wo instruments used during the monitor 
and review visits were thevSchool Levpl Program 
Compliance Review Instrument^^d tj^e Program 
Quality Review Instrument. The contpliance instfu-, 
ment was used at each school visited to determine 
areas in which a -school's program was cAit of 
compliance wi^h regulations. No overall flata were 
compiled" across schools on the extent of compli- 
ance. Thus, the* findings from compliance reviews 
consist of the number of schools that received such 
reviews. ' ' ' [ 

The Program" Quahtv^evieW Instrument was 
redesigned*, for 1976-77. It included 49 items 
related to a school's funded program. ,The items 
were organized into three categories: (Ij develop- 
ment and use of^th^ school's program plan; (2) 
ptogram implementation, and (3) ongoirtg program 
development. Forty -five of the item3Njvere appli- 
cable to programs in non-ECE grades aind schools 
not receiving ECE funding. The rating scale on the 
quality instrument ranged from 0 (no evidence) to, 
5 (excellent). The ratings were avera^ across 
schools to Obtain the information on 'which the 
findings ^Chapter V were based. Copies of both 
the Sch(^^ Level Program Compliance Review 
Instrument and the Pijbgram Quality Review 
Instrument are included in Appendix B. 

Factor analyses were performed on the 1976-77 ^ 
Prtrgram Quajj^y Review Instrument ^information 
to enhance undew^nding pf the qualities it 
Appeared to fne^sure. For grades with ECE fund- 
ing, the strongest factor appeared to be an instruct 
tional factor pertaining to the developmient of the 
plan, diagnosis and prescription in language devel- 
opment, reading and mathematics, continuous prog- 
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ress in^ mathematics, balanced curriculum, and 
i?tJ8rovement of the learning environment. 

W^e same type of f^tor analysis was^erformed 
on the items for grades without ECE fDnding/The 
strongest factor was an instructional/ participation 
factor that included^ the relationship of the 4>re- 
scription to the diagnosis in mathematics, pro- 
viding for students' individual needs through a 
variety of methods and materials, parent participa- 
tion; parent education; staff development for 
teachers., aides^ prinv:ipal, and support staff, and 
ongoing planrmig in the area*'of the evaluation of 
information about the program. 

The factor analyses of the Program^ Quality 
Review Instrument information for both group^ 
indicated that the instructional items-diagnosis, 
prescription, materials, and continuous progress- 
continued to be the heart of the Program Quality 
Review Instrument. Support factors, such as staff 
development, health/auxiliary^3ei=vi4;^, and parent 
participation^ were a second focus, and the needs 
^ of limited- and non-gnglish-speaking students were 
a third^rta of emphasis. ' • 

Secondary school on-site reviews. Secondary 
school programs were reviewed only for compli- 
ance with regulations. The data from the reviews 
.consist of the number of schools that were 
reviewed. 

Evaluation of the Monitor and Review Process 

The Department of Education conducted an 
evaluation of the mpnitor and review process at 
both the elemental^ ajid secondary school levels. 
Questionnaires were sent to each schooUvisited in 
1976-77. A total of 657 questionnaires (53 ojer- 
cent) were returned by the elementary schools^^A 
copy of the questionnake and data obtained from 
it are included in .Appendix C. The responses were 
-generally favorable. Secondary schools ^turned 
956 questionnaires. In general, the process was 
rated as ^satisfactory in the secondary schools. 
Elementary schools returned one survey form per 
school that represented a consensus of viewpoints; 
responses from » individuals were returned from 
secondary schools, which accounts for the fact that 
the number of secondary school forms returned 
was Jarger than the niimber of schools visited. The 
survey form sent fo elementary schools included an 
opportunity for schools to write "recommenda- 
tions." The most frequent recommendations made 
about the monitor and review" process^ were .(1) 
take a longer time in schools/classrooms (20 
percent); and (2) simplify the Program Quality 
Review 'Ijistrument (15 percent). A group of state 
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and district personnel has revised the process for 
the 1977-78 school year.^ 

Advantages and Limitations of the Monitor 
and Review Process 

Both the 'advantages and the hmitations of the 
monitor and review process stem in part from the 
several pujnoses that the process is designed to 
serve. The^idvantages of the on-site review process 
in providing feedback to schools ,about the 
strengths and Weaknesses of their programs include 
the opportunity for persons from outside the 
school or district to provide a different perspective 
of the program. Another strength of the process is 
that it provides a relatively straightforward evalua- 
tion frame woi;k that, in fact, can be\itilized by the 
school itself a^d can be adjusted, if necessary, to 
meet the particular conditions at a school. The 
on-site revfew process is clearly superior to ques- 
tionnaires, surveys, or other documents for gather- 
ing infprmation about program quaUty within and 
across schools. Ho^^eyer, the very strengths pf the 
process-the direct people-to-people involvement- 
limit to some extent the inferences that can be 
drawn from th^ data obtained. One should use 
caution m interpreting the data:.across schools. 

One^ limitation of ratings of program quality 
obtained through the Program ^Quality Review 
Instrument is- the interrater reliability of such data. 
Although every effort is made to ensure that two 
teams applying the instrument at different times at 
different schools are using the same standards or 
criteria when assigning numerical ratings, one 
cannot be sure that differences in ratings among 
schools are not a function of differing rater 
perceptions and standards rather thah actual dif- 
ferences in program quality. The results of a study 
of interteam reliability conducted in 1975-76 and 
reported in Evakiaiion Report of ECE, ESEA Title 
/, and EDY, hQj3'76, indicated that two teams 
visiting* the same school at the same time te^d to 
give very similar r ati| ifi st o the school's program. 

A second limitatioiiSf the monitor and review 
data is that no longitudinal comparison of program 
quality is possible. Both the Program Quality 
Review 'Instrument and the populations bf the 
schools fated have changed frpm year to year, 
making it difficult lo determine whetKer school 
program quality changes as a function of time or 
other circumstances. 

A third limitation stems from the population of 
schools that receive monitor and review visits each' 
year. The schools selected for visits are. either new 
to the program or are those that were ranked low 
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in overall program quality the» previous year. Thus, 
the group reviewed couy not be considered repre- 
.sentatfVe bf^ all schools participating^ in the pro- 
^ grams, and the ratings of tliese "neW" and "low" 
schools 4re not representative of the quality of 
programs across participating schools. , 

Ihdicators of-Student Achievement - 

Student achievement data were derived from 
two sources. (1) California Assessment Program 
tests, and (2) publishers' norm-re fei-e need tests. 

California Assessment Program Data ' ^ 

Attention has shifted in the consolidated evalua- 
^ tion report to utilization of California Assessment 
Prpgram. schoolwide achievement scores. Two 
major advantages of these scores over the school- 
reported publishers' norm-referenced ,test scores 
are that (1) they are based on uniform tests 
' administered throughout the state (the Entry Level 
Test for all grade one students and the Reading* 
^ Test for all students in grades two and three), and, 
(2) they are given only at the time of the year for 
whuJi empirical norms are available. On the other . 
hand, two disadvantages of the California Assess- 
ment Program tests are that (1) they are designed 
to yield only state-, district-, and school-level 
reading data and are hot Resigned to provide data 
on subgroups of Students within schools, akii (2) 
the second and third grade test does not cover 
mathematics achievement. 

Vanous 'analyses were conducted on the basis of 
data provided by the California Assessment Pro- 
gram. Analyses of trends in reading scores anjt'ah 
examination of the relationship^etween leading 
achievement and socioeconomij?^ and dempgraphic 
variables^were conductedJor ECE schools and for a 
matched group of non-ECE schools. This longitudi- 
nal analysis of reading achievement performance of 
students in ECfi" §ghools was based on data 
obtained by complex statistical analyses (multiple 
regression) of tiie performance of students in all 
elementary schools in California. The school served 
as the unit of analysis, and the regressfSnWariables 
included grade one Entry LevehyTest scores from 
the California Assessment Prograhi, socioeconomic"' 
index, percent of bilingual students, and student 
mobility rate from the California Assessment Pro- 
- gram survey. k% a result of ,the analyses, it was 
possible to observe changes in reading test scores 
among ECE schools relative to changes in student 
populations and length of ECE participation. Cali- 



fornia Assessment Program scores are for use in ■ 
only grades two,.threeVsix, and twelve. 

Publishers' Noijn-Referenced Test Pata ^ 

Student achievement data from publishers'^ 
normrrefejrenced tests were reported to^he Det)art- 
ment h^. districts.' These tests measure a broad 
sp,ectrum of skills within identified areas, such as 
^reading or mathematics, but they are relatively 
insensitive to specific instfHctional program objec- 
tives.\They are designed to* reflect the achievement 
gains of 'individual students ||&npared to a large 
population of students, but fflly are not specifi- 
cally geared to the speciaf^eeds of students in an^ 
target group. Consequently, the use of ^uch tests 
tends to result in underestimates of the gains'made 
by. the student toward program objectives. The 
prgblem of njffc uring and comparing the progress 
— eroh 



of schools - 



with its own 



unique ^t^f* 
objectives -was alleviated only partly by allpwmg 
schools to "choose, for use in th^evaluation, a' 
particular norm-referenced t^st fm'm a/list of 
commercially developed standardized achievement 
tests. ,e 

Virtually all participating schools were required 
to adminTstcf publishers' ijorm-referenced tests in 
reading and mathematics on fa protest and post-test* 
basis. Most schools pretested ' students in October 
an4 post-teste|l them the following May; some 
schools teste^on an annual basis (May/May). The 
achievement lests that were used are listed, by^ 
frequency of use, in Appendix D. The. frequency 
distribution of .^lapsed time between prelesting and 
post-testing for schobls is shown in Appwidix E. 

Publishers' national norms for achievement test 
score* represent the achievement levels of typical 
students in the nation. A comparison of participat- 
ing students' scores with publishers' nation3ln9rms 
is useful in that it indicates how patiUcipating 
students are scoring relative to a national sample of 
students at the sanje grade level. 

While test scores have often been expressed in 
grade equivalents, many technical shortcomings 
exist in the use of this particular type of derived 
score.* Given these shortcornings, the Department 
has analyzed student achievement using standard 
scores based on a national mean score of 50 and a 
standard deviation of 10. These standard scores 



*A technical discussion^f the shortvomings of grade equivalent 
scores can be ^Dimd in D. Horst, G. K. Tallmadge, and C. Wood, 
Measuring Achievement Gains in Educational Projects (RMC^ Report 
UR-243>. Los Altos, Calif.. RMC Corporation, October, 1974, pp. 
8-10. ^ 
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were corftputed/ front' mean liw scores when 
inrlevel testing "was conducted ajid 'from' mean scale 
scores when out-of-level testing .was .used.* The 
standayd si:ores^vere computed lis follows 

■(^3 



/t = 50 + I 



ho 



where' T = standard SOore 
' • ^ • X =^chool mean scorQ 

publisher's mean scose • ' 

SD - publisher's standard>^viation 

Conversion of thf data to ^andarcT Vores 
facilitated ii^erpretation of th^ fmdings in relatiori 
to national' norms. Assyme, for exaijriple, that 
students had an average pretest standard sco^ oj^ 
48 and an average 'p9st-test1irSndard score of 48.- 
The kudents would have m'aintained their same 
position relative tcr the*^ national norm group. In ^ 
other words, th? students would have made thc^ 
same gain that the norm group made. To. the 
extent that the posJ-test scpre was greater than the 
pretest^ score, ^he students 'coul.d be considered to j 
hav.£^gaine4 more than the'ngrm group. , * 

* - . In the computation of achievement'gains, scores 
were used for onlj^'tho^ students' for whom both 
pretest and post-.test scores were available. Test 
-information that^as either incomplete or base^ 
'procedural irregularities wdi> not used in developing 
statewide averages. Examples of incomplete dat^ 
and irregular procedures inclu^ied instances in 

, which ( 1 ) either pretest or post-test information 
was omitted', (2) test results were combined for 
several grade levels, \3) scores w.ere^t 
reported in terms 6f either raw scores or scale 
scores; (4) jesl^ without national norms were used, 
(5) the elapsed time betweeif- the •pretest and 
post-test was less than .frve monthsAe) no test 

, results were reported; and 0)j/st scores of 
students who wek not receivl^^^^^ives provided 
for by consolidated application funds were used in 
determining the mean rih score. 

Advantages and Limilations of Student Achievemei/t 
Data Sources • * . ^ /' 

Certain technical limitations\^re con(mon to the 
data sowrces employed in the analyses of student 
-Achieve Q?ent 'gains.'. These include the use of' tests 
;iot specific to program goals, the unavailability of 
test data frbm program partii^ipants only<problertis 
in interpretation as a rftult of'' estimated rather 
than empirical norms, and the inapprc^riateness of 
certain evaluation designs. 

Jhe major, advalitages of , school-reported pub- 
lishers' norm-referenced test scores are that (1). 



^tricts to address unique district and school 
^ concerns; and (2) they are the only achievement 
score«?available to assess the progress of students in 
grades not tested by the California Assessment 
Progrart. The important disadvantages of school- 
report^ publishers' norm-referenced |es't scores' 
are that (1) major differences exist among the 
available standardized tests. in t^rms of what they - 
measure and how they describe tho- progress of . 
students from grade to ^ade;^nd (2) many of' 
these^ teists^hav£ norms that may, because of the 
e^iiaption procedure used'in the calculation oftthe . 
- nOrm$ throughput the school' year, pi;esent an 
overly optimistic picture of participating students' 
,^ogres^elati9^ to that of the -national norms. 
• Despite^ese tedinical iHsad van tag^sf publishers'- 
norm-referenced test scores |re particularljTimpor- 
tant for dittrict- affid school-le^el evaluations and 
do provide useful iftformation at the st^te level ^ 
about i^e progress of students? 

The most accurate ^orm scores^ for publishers' 
nOfm-ftfereted tests kre those for the time of the 
yearijMing alhidh the publisher's original norming 
stud/ ^a§ jfcade. The norm Scores for other times 
dyiing aXis^ah-^ctually estimates obtained by 
interpolating across grade levek. In most cases^he 
. norms are underestimates ^of the 
growth in sjtfedent igiievement between the 
and spring of^ year^ a consequence of this 
inaccuracy, there js a tendehcy for students taking • 
the publishers' nornvrefe;^cexi tests to appear to 
Q be g>rogressing faster Jhan, the norm group betw^een 
the fall and spring. Unfortunately, no fully satisfac- - 
tory mefh^^ exists for adjusting the publishers' 
norm-refer^flced test' to ayow for , this influencs.^ • 

The determination of the mean weighted average 
standard scores for att participating students at 
> each grade level required that^ certain conditions be 
imposed. First, jnatched scores were omitted for 
hose students for whom the Jjiterval betw^n the 
Tetesf ana the .post-test .was le'ss than five months, 
^his practice resulted in the elimination of data for 
IJcrcent of those students W whom pretest and 
^)ost-test s^res were available. The 4maining data 
were aggregated regardless of testing ftiterval. 

>n analysis was m'ade.of test score differences 
associated with annual testing (usually May to 
Maj^r and within-year 'testing (usually October to 
May). For a majo^ty (66 percent) of the studente^ . 
for whom test score^ were reported, the interval 



K-^nternelated' 



they are based on .tests that 



were 

A 



^chosen by 



• ^ A technical discussion of interpolated norms can be found in D. 
llorst, G K. Tallmaage, and C. AVood, Measuring Achievement Gains 
ill Educational Projects (RMC Report UR-243). Los Altos, Calif • 
RMC Corporation, October, 1974. * ' ' 
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Jjjstween 'pretesting and^post-t^ting was from 6.5 
to 8.5 months (within-yeaj* testing). For 24 percent 
the interval was between 11.0 and 12,5 months 
(annual. testing). Figure IV-1 shows the differences, 
j^not adjusted for unequal pretest means, in average 
standardized gain scores, by testing interval, for 
fluent-English-spe^ng students, 'i^though they 
were consistent with the overall results, the data 
for grade one were omitted because' not all schools 
that were testing oo an annual basis pretested grade 
one ^tildents ^at t;he end -of kindergarten. In 
addition, the increasingly .smaller number of 
schools testing^ students in gfades Severn through 
tWelVe witjiin the "specified testipg intervals sug- 
gests that one should exercis^ caution in inter- 
't)reting differences.' pie general^ finding^of greater 
gain scores for students tested within the school 
year than /for those tested ahnually is consistent 
with othCT findin^^jhat suggest inflated gain 
e§^imates' for this gi:6aip of students. ' » „ 

Xhe- data in Figure IV* 1 sho\v that at all grade 
levels, fluent-English-speaking students who were 
tested within the school yeaij made gains* apcj^ that, 
the greatest average standard sbore gairt (4.1) was^ 
in gr^dVthree. The annual reTu^s. indicate overall 
average declines in grades four, sfeven, aiid nine and 
gains at the other grade levels* <)nly in grade eight 
was more g-owth -j^ote^ f^ students tested 
' annually /than* for those. testS^>^thin the school 
^year. No plausible explanation cant)e offered for 
.this occurrenc^. Neither is any explanation offered 
for the gpft^ter fluctuation across grade jevels of 
gain scofes ^om annual testing than of those from 
within-y^r testing. Heretofore, the annual testing 
pattern .had'ljeen thought to,yield-a stable measure 
of growth in achievement^ .* t 

A principat qi^estion in interpreting the achieve- 
ment of pat^fcipating student? is: How well did 
students, perform compared to how th^ would 
have g^rtormed had they not participated in 
mtiltiple-funded school programs? This question 
cannot be ans>veifed 'djrectly because .districts, 
schools/^nd stuHents are ^elecfted for participation 
in pSEA Title J, EDY, bilil^al education, 
and Miller-ynfti(^ programs Jn accordance with 
prescribe^ eligibility cnteria. The effect of these 
selection ffrocedureS is that particip^ing students 
differ ^ffeiiTi nemparticipants not only in terms of 
the critdna for eligibility that^they meet but also in 
ternis of other variables related to school back- 
groimd aftid sttident population characteristics. 
Consequently, evaluators have develpped tech- 
nique^Nthat provide indirect evidence from which 
inferences can be mad^ about program effects. In 



' this equation report inferences have Been made 
on the basis of two types of comparisons. (1) those 
between program participants and publishers' 
national norm groups, and (2) those among ECE 
schools and between ECE. schools and non-ECE 
schools .on the basis of California Assessment 
PrograiHi data. 

The use of the sam^ student achievement data 
for selection of participants and for assessment of 
student growth distorts the interpretation of stu- 
dent outcome daja for two reasons. First, a 
phenomenon of statistical variation in test scores 
occurs when the same test is used to test the same 
students more than once. Second, those circum- 
stances related to the^lection^of particular stu- 
dent participants may not be the circumstances 
addressed in the program, that is, student -test 
performance may be a function of vanables other 

• than 'those addressed in the program. These latter, 
variables may continue to affect students during 
their participation in the, funded programs. 

^e procedures for allocating Title I and EDY 
funds \o schools are different, but in the allocation 
of both types of funds, the same schools often are 
identified as eligible for funding. Schools may be 

* eligible Tor Title* f fu'nds on the basis of the poverty 
of ^tudents^' families and for EDY funds on the 
basis of low student achievement. When a district is 
distributing Title t and ^EDY funds, the decision as 
to which school receives which funds is often 
arbitrary. Thus, students in schools receiving only 
Tit^"^ funds are likely to be quite similar to those 
in sfchools deceiving only EDY funds. 

While fairly accurate generalizations can be< 
made .about funding patterns in schools svithm a 
Single" district, direct companions of funding 
patterns in school^ from different district^ are 
unjustified because of the nature of the selection 
process in each district. The school witli the lowest 
achievement in one district ma> actually perform 
- at a mtich higher tevel than the school with the 
lowest achievement in another district with differ- 
ent population characteristics. Nevertheless, when 
reading the section on school program charac- 
teristics and interpreting the findings jji Chapter V, 
the reader should keep in mind those differences in 
school characteristics that are a function of fund- 
ing source eligibility.- 

A significant limitation of this and other efforts 
to utiUze achievement measures in the evaluation 
of program outcomes is that achievement data are 
a single and^ sometimes crude estimate of educa- 
tional effect. Stud,ent achievement data represent 
academic outcomes only and leave unmeasured 
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•Insufficient data for computation ! 

NOTE: Numbers in parentheses arc the numbers of schools that rcponcd test data. 

Fig IV L Changes in average standardized reading scores for fluent-En^ish-speaking students for 
^to post-test intervals of 6.0-8.5 months and 1 1.0-12.5 months 
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other important educational outcomes, sucti as 
improvement in self-concept and personat and 
social growth. 

In addition to the weakness of using a single 
indicator of educational outcomes, another mis- 
take that "6 frequently made is the attribution of 
single effect outcomes, such as achievement gains, 
to jiarticular **causes," such as program participa- 
^Tlbn^The inappropriateness qf estab'ihhing cause- 
effect relationships is a senous limitation on the 
interpretation of all student achievement data in 
this report. 

\ In 19^6. the State Board of Education asked the 
Educational Management and Evaluation Commis- 
' sion Evaluation Committee for counsel regarding 
the appropnateness of .cross-program comparisons 
in edu<:ational evaluation, such as might beacon- 
ducted if one were attempting to examine the 
relative ments of EDY programs versus bihngual 
education programs, versus ECE. for example Fiv^ 
recognized evaluation experts were invited to 
prepare written comments on the issue 

The Commission considered the following major 
points. (\) A distinction is made between funding 
sources and programs. Funding sources in and of 
themselves do not define edjjfiistional treatments 
and, consequently, do not.n^e direct impact on 
children. The result is that when attempts are made 
to evaluate a particular educational program, it is, 
impossible to relate the characteristics of the 
program to the specifications of the funding 
source, (2) The overlap and **mteraction*' of- 
funding sources at man> schools make it impos- 
sible to determine Jhe effects of the particular 
sources. Funds are accounted for separately , but 
services are comminglethvthereb> , making it impos- 
sible to determine the unpact of specific funding 
sources. (3) The dissimifarity ^ the student popu- 
lations receiving services makes meaningful cross- 
program comparisons impossible. The commission 
subsequently submitted to the State Board a 
majority report in which the authors concluded 
<hat under current circumstances, meaningful 
cross-program comparisons of the major educa- 
onal programs inXalifomia are not feasible ^ 

Studies of the Rehtionship Between Monitor and 
Review and California Assessment Program Data 

The purpose of this section is to examine the 
relationships between the ratings that schools 



"Coruidenng the Possibility of Meaningful Cxoss-Prognm Com- 
pamoiu.** A posauon statement prepared by the Educational 
Management and Evaluation Commission Evaluation Commute^ and 
submitted to the Sute Board of Edu«;ation on November 11^976. 



received on the Program Quality Review Instru- 
ment and the data obtained through the California 
Assessment Program. The limitations of each type 
of. data must be considered, and the relationsfiips 
or lack thereof between the data n^ust be inter- 
preted with great cautidn. 

When Cabfomia Assessment Program scores are 
used for program evaluation, the cntenon used is 
the standardized, residual scored (the difference 
between actual and predicted scores) based oji 
grade three results on the 1976 Reading Test. 
School background factors are considered in the 
determination of these scores, and the scores are 
scaled so that a school's score (-average score jof the 
pupils in the school) can be compared with those 
pf similar schools. One of these background 
factors, socioeconomic index, has a modest associ- 
ation (r*= 0.23) with the Program Quality Review 
Instrument score. This may be a reflection of the 
ability, of schpols with high socioeconomic index 
ratings to provide belter^ programs. No jelatioip 
ship IS nQted_ between Cabfomia Assessment Pro- 
gram residual scores and the socioeconomic index. 
This IS expected because the socioeconomic index 
was one of the background "factors used in the 
score scaling. 

Asa follow-up to the 1975-76 correlation study 
between the Program Quality Review Instrument 
ratings and Catlfomia Assessment Program scores, 
Pearson product moment correlations were deter- 
mined betw€en%ie actual, predicted, and residual 
reading scores from the California .Assessment 
Program and items on the Program Quaht> Review 
Instrument. The Program Quality Review Instru- 
ment was analyzed by subsections to determine the 
relationship among reading achievement, the 
mstructional factor oh the instrument: and the 
four Items that dealt speafically with aspects of 
the reading program. Table IV-1 shows the correla- 
tions for ECE schools^ 

In ECE schools almost no relationship was noted 
between the Cabfomia Assessment Program^ata 
and the Program Quabty Review Instrument data. 
Even when specific readmg items from the Program 
Quabty Review Instrument were correlated with 
reading achievement, the relationship \vas still 
weak, ranging from 0.tf2 to 0.12. 

The fact that monitor and review scores have no 
correlation With test scores does not imply that if 
one IS a measure of program quality, then the other 
is not. If two variables measure different aspects of 
a third variable,, then 'the first two may have no 
correlation with each other but may have a 
significant correlation with the third' vanable. In 
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TABLEIV-1 

toirelations Between California Assessment Program (CAP) Third Grade Reading Scores 
and Program Quality Review Instrument (PQRJ) Ratings for ECE Schools, 

1976-77 . . ' • 





CAP actual 
score V 


CAP predicted 
score ^ 


Residual score 
(CAP) 


Total score, 
PQRI 


Total score, PQRl 


-r0.04 


-0.01 


" -0.05 


+1.00 


Instructional factor, PQRI 


-0.06 


-0.04 


-0.05 


+0.90 


PORF mimic inctnirtirmal fartnr 






+0.03 


-*0.43 


PQRJ reading component* 
Diagnostic assessment 

Relationship to diagnosis , 

Learning styles 

Continuous progress / 


* 

+010 
+0.11 
+0.15 

r 

+0.12 
1 * 


+0.10 
+0.07. 
+0.12 
+0.12^ 


+0.02 
+0.08 
+0.09 
+0.04 


+0.66 
^+0.52 
+0.72 
+0.28 



*FQRJ reading component refers to items 1. 2. 3, and 4 oh pages 3 and 4 of the Program Quality Review Instrument. 



this case, the Department is trying to assess 
**p'rogram quality " Thus, although monitor and 
review scores and keading Tesr, scores are measures 
of two entirely different aspects of program 
quality, the use of a combined measure of both 
monitor and review scores and Reading Test scores , 
to describe program quality is appropriate. 

Special Studies 

Each year the Department of Education under- 
takes special studies in response to program or 
funding questions raised in previous evaluation 
activities. Ip the 1975-76 c'onsolidated evaluation * 
report, a decline was noted in student achievement 
residual >scores among- schools in the lowest 20 
percent of the distribution of scores on the 
California Assessment Program Entry Level Test.^ 

A special case study was designed to investigate - 
the unique relationships between student achieve- . 
ment and certain school characteristics at 1 6 ECf ' 
schools selected from this group. This study was a • . 
cooperative effort by Department staff from the 
Office of Program Evaluation and Research and * 
from Elementary Education Program Management. 



Half of the schools had made gains in grade three 
reading achievement in the preceding two to three 
y^ears, and half had experienced declines. 

Case studies were prepared for each of the 16 
schools by pairs of observers who spent four days 
observing in classrooms, interviewing school per- 
sonnel, and reviewing documents in each school. 
The focus of the observers' visit was the "recon- 
struction" of events', that cou^d be related to . 
changes in pupil achievement. The teams attempted 
to develop for each school the most plausible 
explanation for the changes in student achievement. 

A companion. study was conducted under con- 
tract to the Department by SRI International, 
Menlo Park, to identify classroom processes related 
to changing test scores among schools in the lowest 
20 percent on the Entry Level Test. Trained 
observers recorded observations about materials^ 
activities, the organization of groups, the number 
of teachers and aides involved in the mstructional 
process, instructional strategies, an^ the behavior 
patterns in 45 classrooms in 14 schools. Half of the 
schools in ^he study were also included in the 
special case studies conducted by the Department 
of Education. 



residual score is the difference between a predicted score and 
an actual score. 
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V. Findings 



The Tindings reported in this chapter provide the 
basis for inferences about the ^extent of participa-; 
tion ih^ funded programs, the allocations and 
, expenditiires^in the funded programs, the quality 
of programs and services, and the achievement of 
students who received services through the various 
funding sources. Additional findings relevant to ^ 
early 'childhood education schools and selected , 
findings that are characteristic of local evaluation 
efforts are reported in chapters VI and^/VII, 
respectively. ' 

The findings '^reporte^ below include enumera- 
tion data on program participants, a comparison of 
background characteristics of participating and 
noiiparticipating students, indicators of program 
quality, and student achievement data. Student 
achievement results are presented first for analyses 
based on California Assessment Program data and 
then for analyses based on publishers' norm- 
referenced test data. Information about schools ' 
engaged in restructuring (ECE schools), schools 
that addressed the special neeSs^o^ identified 
students, and schoojs thaf received funds from 
multiple sources is included. 

« Enumeration Data on Participants 

Enumeration data on sttidents, parents, volun- 
teers, school and district participants, and program 
expenditures 'are repo^d in this section. These 
data have been a^egated for all schools funded 
through the consolidated application. Student 
. participation data are also reported on the basis of 
funding source. 

.Student Participants 

As shown* in Figure V-1, a total of 1,133,729 
students in kindergarten through grade twelve^ 
jiarticipated in school programs supported by ECE,^ 
ESEA Title I, EDY, Bilingual education, and/or 
Miller-Unhih funds. This figure represents ^ 21 
percent increase in student participants since 
1975-76 and a 41 percent increase since 1974-75. 
Almost all of the increase in participation is 



accounted for by the expansion of ECE, in which 
the percent of K-3 students iJarticipatin^ more 
than doubled from 1974-75 to 1976-77. Partici- 
pating students represented 27 percent of the total 
kindergarten throu^ grade twelve enrollment in 
California public schools. Of.thoM students who 
were program participants, approximately 72 per- 
cent were enrolled in kindergarten through grade 
three, 20 percent were enrolled in grades four 
through six, and 8 percent were enrolled in grades^ 
sevto through twelve. Of the program ^participants 
13 percent were limited- or ijon-English-speaking 
students.' The number and percent of program 
participants who were limited- or non-English 
speaking and the total number of participants for 
each funded program are displayed in Figure V-2. 

Spanish was reported as the primary language of 
15 percent of all student participants. Chinese, ' 
PiKpino, and Native American were reported as the 
primary language of 0.5, 0.4, and 0.3 percent, < 
respectively. English was reported as the primary 
language of 82 percent of the student pajticipants. 
Eleven other languages were represented by the 
remaining 1*8 percent of the student participants. 
' During 1976-77 approximately r,070,000 stu- 
dents participated in the reading components, and 
1,066,00.0 students participated in multicultural 
education components.. The language development 
and mathematics components served 1,043,000 
and 1,039,000 participants, respectively. Health 
and auxiliary services were provided for. 982,200 
students.. Optional components, sucH as psycho- 
motor development, music education, art educa- 
tion, and career development, were provided /or, 
3^81,750 students. These numbere are duplicated 
counts; that is, participants in more than one 
program cohiponent were cbuhted in each 
component. , ' ' 

School and District Participants 

In addition to participating in instructional 
activities*' school and district personnel partic-i 
ipated extensively in staff development activities'^ 
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Total enrolLfnent for each gradi 



X:h grade^ 



^ Number; of participants for each grade 



Grade 




8 

> 

9 
10 
11 
12 




300,512 



79,842 



.306,844 



76,932 



322,865 



/70,699 



344,250 



31,966 



^18,137 



346,471 t 



1,547 



Total K- 12 

enrollment = 4,235,525* 

Total number of K-12 
students served = 1,133,729 





• 364.471 




1 


23,479 




366,299 






— r 1 








336.067 


i 3,753 










285,868 
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^Souxce - Active Enrollment in California Elementary and Secondary Public Schools. Sacramento. California State Department 
of Education, 1976. 



Fig. V-I . Number of students enrolled statewide and number of student participants in programs supported by 
consolidated funding, by grade level, 1976-77 
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1 5 percent 



10 percent 



100»261 



19 percent 



20 percent 



46,065 



27,413 
60 percent 






^ Total nurpber * 
of participants 



Number and 



fbercenll Percentage of total 
ipercentj ^j^^ were NES/LES 

' (non-English-speaking or 
limited-English-sj)eaking) 



NOTE. These numbers represent duplicated counts. A student .who received services provided for by ECE 
funds and by EDY funds, for example, would be counted for both funding source combinations. 

I Fig. V^2. Numbers of limited- and nonrEnglish-speaking students participating in ECE, EDY, ESEA Title I, 
bilingual education, arid Miller-Unruh programs and percent each group was of the total program " 
participants, 1976-77 '^'^ ^\ ^ . " 
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Through the consolidated funding sources, a total 
of 100,100 personnel received some staff develop- 
ment training. The previous year 87,530 indivi- 
duals were reported as having received staff devel- 
opment training. 

Parent Participants ^ 

^e participation of both parents and volunteers 
varied from month to month. During the sixth 
month of instruction, schools reported that 
432,200 parents were involved in the parent 
participation and community , involvement com- 
ponents. Parent ."education was provided for 
292,700 adults, and |93,900 adults participated in 
the health and auxiliary services condone 
counts were all duplicated cqunts. 

Volunteer Participants 

During the sixth morjtfe^f instfuction, a total of 
about 131J0fl....--adtm volunteers contributed 
1,327,900 hours of assistance to the school pro- 
grams. For the same period 90,300 cross-age 
volunteers assisted 7or 1,063,000 hours. Thus, a 
total of about 228,000 Volunteers worked in the 
schools in 1976-77, as compared to a total of 

81,000 volunteers m 1975-76. 

** • • • • 

Student Participants, by Funding Source 

The following sections con:tain data on student 
participants, by funding source. 

ECE participants.^ A total of 2,455 schools in 
829 districts received ECE funds in 1976-77. These 
schools reported 656,500 student participants in 
ECE, compared to 42^,700 in 1975-76. Of the 
participants 163,359 were in kindtergarten, 
173,700 were in grade one, 164,000 were in grade 
'two, and 155,400 were in grade three. During the 
years 1973-74 through 1976-77, the percent of 
kindergarten through ^ade three students state- 
wide who participated in ECE increased steadily. 
Table V-1 shows the percents of students in these 
gi;ades statewide who participated in school pro- 
-ams funded by ECE for the past four years. 

EDY participants. During the 1976-77 school 
year, 43270OO students, ffom prescJipoT through 
grad^ twelve, participated in programs funded by 
EDY , or by EDY in combination with other 
fundmg sources. Approximately 400,700 students 
participated in such programs in 1975-76. Of all 
students served, 53 percent were enrolled in 
kindergarten through grade thrpe, 33 percent were 
enrolled in grades four through six, and 14 percent 
were enrolled in grades sever\J|j-ough twelve. The 
number of EDY participants, by grade level, is 
shown.in Appendix F. 



TABLE V-l 

Percent of Students Statewide Participating in ECE 
Programs, 1973-74 Through 1976-77 



Percent of ECE enrollm^ 
by school year 



Grade level 


1973-74 


^974.75 


1975J6 




Kindergarten 


17.2 


25.3 


, 34.4 


52.9 


One , 


16.6 


25.5 ^ 


33.9 ^ 


51.9 


Two 


11.4 


23.9 


33.6. < 


' 51.7 


Three 


9.9 . 


' 23.1 


33.6 


51.6 


Total 
\ enrollment 


13.8 


24.5 


^4.0 


52.0 



'SEA fitfi I participants. During the 1976-77 
school year; 529,400 students from ^>reschool. ' 
thiough grade twelve participated in ESEA Title I 
prd^ams. During the previous year 529,890 stu- ^ 
dent^ participated in suph pro-ams. Of the totjal 
number of students who received services, 5 15, 400 
were enrolled m public schools. Detailed enrolW — s 
ment data are {Presented in Appendix G. Consistent-' 
with state regulations, th^^'^^^te^ concentration 
of participants was in the prirnm^ grades, where 55 
percent of the participants were served in kinder-, 
garten through grade three. Table V-2 shows the 
percents, by grade levelf of students in CaUfornia 
Who received ESEA Title I services from 19^7-68 
through 1976-77^ ^ 

Participation data on special compensatoi:y edu- 
cation programs funded^under ESEA Title I, such 
as programs for handicapped students and for 
neglected^ and delinquent youth, are provided 
elsewhere in this chapter along with findings about, 
those programs. 

Bilingual education participants. The total 
number of students who participated in state 
bilingual education programs funded through the 
consolidated application, was 46,000. The figure 
for 1975^7^ jwas^ 49^000. Of th^ students wha 
participated in bilingual education programs, ^60 
percent ^ere classified as limited-EnglTsh speaking 
or non-English speaking.^ The number of parti- 
cipants, by grade level and by level of English 
fluency, is shown in Table V-3. * 

Miller-Unruh, participants. Participation figures 
for^ Mi'ller-Unruh were estimates derived from 



Students were classirie4 as limited- or non-English speaking on 
the basis of district language fluency surveys conducted in the fall of 
1976. 



ERLC 



34 . , 



TABLE' V-2 

Percent of Students Receiving ESEA Title I Services in California, by Grade Level 
^ , Groups, 1967-68 Through 1976-77 



Percent of total ESEA Title I enrollment, by school year 



Grade level 


1967-68 
1— 


1968-69 


1969-70 


1970-71 


1971-72 


1972-73 


■1973-74 
. I 


1974-75 


1975^-76' 


1 


)76-77« 


Kindergarten 














1 










through grade 


















53.4" 




three 


40.4 


41.8 


50,4 ■ 


52. 1 ' 


51.9 


54.0 




54.3 


5A.6 


Grades four 












35 


V » 








through SL\ 


22.8 


23.7 


33.0 


33.9\ 


34.7 


°30.3 


34.0 


33.8 


'32.6^ 


Grades seven 






















through nine ' 
Gmes tei) 


- 19.9 


20.7 






, "8.2 


' 6.f 


8.4 


' 8.3 


9:4^ 


9.5 


through twelve 


12.4 


10.9 


,4.0 


3.6 


.3.3 


2.7 


' ' 3.7 


■2.6 


2.9 


2.8 



* NOTE: Figurei for participants in preschool and ungraded pro§j»tm are not included in this table; therefore, Ihe values in 
the res{)^tjye columns do not, total 100 percent. 



TABLE V-S 



^mber of Students, by Grade and Language Facility, 
, in State Bilingual EducationPrograms^ 1976-77^ 



Grade ^ 



Language facility 


K 




'2 


'3 


4 


. 5 . 


6 


• 7 ■ 


• 8 


9 


10 


11 


12 


•Total 


"Non-English speaking 


3,799 


2,0(1 


924 


" 752 


372 


305 


•2^7 


590 


401 


5lC 


248 

J 


131 


^6 


-10,413 


Jimited-English^sp^a|dng 


— < 

2,480 


1 

3,372' 


2,710 


•2,013 


1,165 
^ 


1,066 


_ 871" 


968 


868 


944 

-1 


617 


440 


237 ^ 


17,801 


Qther * ( . 


3,594 


3,218 


2,803 


2,628 


1,488- 


^^391* 


1,100 


'407 


344 


352 


208 


135 


' 194 


17,851 


Total ^ . . 


9,g73 , 

V 


^ 8,656 


6,437 


5,3b' 


3,025 


2,762 


2,2(^8 


1,965 


1,613 


1,857 


1,073 


706 


497 


46,065 



reported particfpation.. The estimated number of 
students served was 135,^60, compared to an 

^ estimated 154,000 students in 1975-76. New legis- 
lation in 1 976-77 permitted students in gradesYour 
through six to receive services. About 0.3 percent 

4 of the students who received Miller-Unruh services 
were in grades four through six. 

Special Analysis of Background Characteristics 
. ' of Student Participants 

A sample of California Assessment Program 
pupil information sheets was examined to deter- 
mine the background characteristics of students 
who participated in programs that received ESE, 
EDY, ESEA Title- 1, bilingual' education, and/or 
Miller-Unruji /unds. Thd^e- data, wl^Kch illustrate 
some of the aifferences in the background charac- 
^ teristics of participating and nonparticipatii)g stu- 
dents, are displayed in Table V-4. ^ 

The occupation of the principal breadwinner in 
a student's family is an estimate of the socio- 



economic statul of the family. Teachers indicated 
what they knew or believed to be the oqcupation 
of the pmicipal breadwinner in each student's 
fantily. A sqialler percent of the pajqtic^a^ing 
students than nonparticipating students cameTrom 
homes in which the principal breadwinner was a 
professional or semiprofessional, and a larger per- 
cent of participants than nonparticipajits came 
from homes in which 4he breadwinner's obcupation 
was classified as unskilled or was unknov 

For the purposes of the Califorma^ Assessment 
Program, English-langtiage fluenpy is determmed 
on the basis<i!i)f the judgment of ^ach student's 
teacher about the student's langUage background 
and skill. ^ Most student parucipants and non^ 



The California Assessment Program determmation of English 
language fluency was not designed to identify the English-speakmg 
^bibty oN^dividual students. As a result of AB 1329 (1976), a 
home language, survey is conducted each fall to determine the 
English fluency of all students enrolled in California public schools, 
, in kindergarten through grade twelve. 



.i TABLE 

Pupil Backgfmind Characteristics of Participants and Nonparticipants, from a 2 Percent 
Sample of 1976-77 California Assessment Program Third Grade Pupil Information / 



Background characteristics ' 



Occupation of principal breadwinner 
Executives, professionals, managers 

Semiprofessionals, cleridTand sales workers, 
technicians ^ 

Skilled and semiskilled employees 

Unskilled emjiloyees (and welfe^) 

Unknown 



pnglish'language fluency 
Englfsh only 

English and another language 
Limited-Engli^ speaking * 

Mobility/ transiency^ ^ 
Enrolled sinde the previous year 

Enrolled in the year of the test 



Percent of participants 
(NumUer.of students = 
3,552) 



13.6 




38.3 

21.7 

6.8 
100.0 - 



75.3 



16.2 

8.5' 
100.0 



94.4 

5.6 
100.0 



Percent of norlparticipants 
(Number of students - 
^ 1,944) 



#2.5 

27.i 

36.1 

9.9 

4.4 
1Q0.0 

87.7 

9.8 

^5 
100.0 

95.3 

4.7 
lOOj} 



ERIC 



^^7 



{5articipants in the sample spoke English only. A 
greater percent of participating students (16.2 
percent) than' noagarticipating students (9.8 per- 
cent) ^'poke English 5nd another Mnguage. The 
perce/(t of participants who were limited-English 
speawing was much larger (8.5 percent comj^red to 
2.5 /pereent) than the percent of nonparticipants 
who. were limited-English speaking. The reader** 
should also note that students who do not speak 
Engli^ (non-English speaking students) do not 
take th<^ Californi5_ Assessment Pfogram reading 
achievement test and therefore were not included 
in this sample. The mobility or transiency af 
participants was slightly higher than that of non- 
participants. Of the students included in the ' 
sample, ^.6 percent of the participants were 
enrolled in the school for the first time in the year 
in which the test was'giveji (1976-77); 4.7 percent 
oHhe nonpart^ipants were enrolled in the school 
for the first time in the year in Vhich the test was 
given. The differences between participating and 
nonparticipating students indicated that the usual 
" statistical criteria for making comparisons betw.een 
two groups were not riiet, and thus comparisons 
^were not justifiable. 

Allocations and Exjlenditu^es * 

The total allocations for the past three years /or 
school progams supported by ECE, EDY, ESEA. 
Title I, bilingj;^ education, and'Miller-pnruh funds 
are shown in Table V-5. Tn the 1976-77 school 
year, 57 percent of participating schools received . 
funds from two or more sources, ^ 



Districts are required to file complete financial 
reports regarding funds received through the con- 
solidated application. The data in Table V-6 are 
from ,a random 'sample of preliminary financial 
VQpjbvti filed by districts. Table V-6 shows the 
percents of tot^l expenditures in seven budget 
categories*, in the districts whose reports were 
included in the sample. \ 

Examples of the types of classified personnel for 
whom funds were expended include classroom 
instructional aides, clerical support, and com- 
munity aides. I^esource teachers, reading or other 
curriculum specialists, n^es, counselors, psychol- 
ogists, librarians, and program directors are , 
examples of the kinds of certificated personnel for 
whom funds were expended. 

Expenditure patterns for each funding source 
over the past three years are discussed in the 
.following section. Because the patterns are based 
" on samples of preliminary reports knd because they 
ipTy reflect school level choices about which 
source to use to provide specific program services, 
thg reader is adyised*to use caution in making 
interpretations. For all funding sources, expendi 
tures must be made to provide services that supple- 
ment, not supplant, jhoS^iirf^the basic programs. 

Early Childhood Education 

Chapter^ 1 147, Statutes of 1972, prpvided for an 
appropriation of $25 million in 1973-74 and $40 
mDUon in 1974-75 for ECE. -Since 1974-75 ECE 
has been expanded annually througli funds pro- 
vided in the state budget. As is shown in Table V-7, 



TABLE V-5 ^ 
Allocations of ECE, EDY, ESEA4i*tf I , fiilingual Education, 



■Type*, of funds 


1974-75 i 


1975-76 


1976-77 


Federal funds 








ESEA Title I* 


$132,577,018 


' $130,000,000 


$136,827,640 


State funds 








ECE 


41,000,000 


61,894,358 


97,405,224 


EDY 


84,000,000 


^0,389,376 


97,411,666 


Bifingual education 








(AB 5284V 


3,836.000 ' 


8,139,000 


8,09 kl 37 


Miller-Unruh 


15,349,625 


13,849,625 


^13,849,625 


^ Total 


$276,762^3 


$304,272,359 


$353,585,292 



ERIC, 



^77208 



♦These figures do not, include ESEA Title I Handicapped, Negledted and Delinquent', or Migrant funds. 
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the 1976-77 state budget provided for $97.45 
million for the support of ECE. These funds made 
possible the extension of ECE to approximately 52 
^-^ercent of California's student population k| 
^^ndergarten through grade three. Table V-7 also 
reflects the expansion of ECE from school year 
1974-75 through school year 1976-77. 

Table V-8 shows the pattern of expenditures of 
,ECE funds over the past three years. The percents 
expended in the budget categories shown remained 
basically stable over the three-year period.. The 
data show , that schools expended the largest 
percent of their allocations on salaries for classified 
t personnel, the second largest percent of expendi- 
tures was for salaries for certificated-^personnel. 
Less than 30 percent of the total ECE expenditures 
were for employee benefits; boote? supplies, and 
equipment replacement; contracted services; 
capital outlay; and indirect costs. ^ 



/ 

The Department is currently examining possible 
reasons for the changes from previous years' 
expenditure^attems. 



pf students 
appropriated 




fable V-1 1 displays the numbe 
served by ESEA Title 1 and the fund, 
from 1974-75 through 1976-77. 

-> C 

TABLE V.7 

Number of Students Served byJ^^ and Funds 
Appropriated/ 1 974-7S^ough. 1 976-77 




TABLE V^6 

Budget Categories of Consolidated Application 
Funds, by Percent of Expenditures, from 
Random Samples of Preliminary District 
Financial Reports, 1976-77 



Year 


Numtjer of 
K-3 students 
served ' 


Percent of 
K-3 students 
statewide 


Funds 
appropriated 


l*>74.75 


303,100 


24.5 


$40,000,000 


I975.7(> 


426,700 ^ 


P 34.0 


63,200,000 


1976-77 


656,500 


52.0 


97,450,^000 



Expenditure category 


Percent of total 
expenditure 


Classified salaries 


54 


Certificated salaries . 


22 


Employee benefits / 


14 


Books, supplies, and equipment 




replacement 


6 


Capital outlay^ ^ 


1 . 


Contracted services 


2 


indirect costs 


1 


Total 


100 



TABLE W'S ^ 

Budget Categories of ECE Funds, by Percent of 
Expenditures, from Random Samples of 
DistrictJReports, 1974-75 Through 



4^ 



Educationally Disadvantaged Youth 

Table V-9 shows the number of students served 
by EDY and the funds appropriated from 1974-75 
through- 1976-77. 

The data in Table V-10 show that the EDY 
expenditure pattern in 1976-77 differed from the 
expenditure patterns in the two previous years. 
Salary expenditures in 1976-77 were equally 
divided between classified and certificated person- 
nel salaries, /i second random sample of district 
reports yielded a similar patte^i of expenditures. 



Expenditure 
category 


Percent of total expenditure 


.197475 


1975-76 


1976-77 


Classified salaries j ' 


'\ 55 


45 


57 


Certificated salaries 


• 20 


28 


21 


Employee benefits 


, 11 


9 


8 


Books, supplies, and 




■% 




equipment replace- 








ment 


■ 8 


9 


9 


Capital outlay 


.3 


2 ^ 


1 


Contracted services 




5 


X 3 


Indirect costs 




2 


1 


Total 


109 


100 
# ' 


100 



ERIC 



The data in Table V-1 2 ^J«y)v no substantial 
^differences in the percents of expenditures, by 
budget category, over the Jhree school' years. The 
percent of Title. I funds ejcpended for classified / 
personnel was lower than the percent of ECE funds 
expended for such personnel. In all three years 
approximately 90 percent of the total allocations 
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^nded for staff salaries and employee 



were ex 
benefits. * 



Bilingual Education 

Tabk V-13 shows the numbers of students, 
served in bilingual, education progr^s and th 
funds appropriated from" 1975-76 through' 
1976-77.- 

Comparative data on expenditurfe patte^ms for 
bilingual education were available for the past two" 
years only. Taple V-14 shows jtlie^milarity of 

• • ^ ■/ 

TABLE V?9 

Number of Students Served by EDY and Funds 
Appropriated. 19J4-75 Through 1976-71[^ 





*Year ^ 


Number of 
^ students 'served 
(all grade levels) 


, Funds 
• — ap^^priated 


r)74.75 
r)75.76 
im-77 


• 389,500* 
, 400,700 
432,000 


$83,754,000 
90,389.376 
97,480,000^ 



•In the Evaluation Report ofECE, ESEA Title /, and EDY, 1 ^74- 75, 
the number of EDY participants reported (436^)00) represented 
count of all students in schools that received EDY funds. For fiscal 
years 1975-76 and 1976-77, the summary counts reflect the actual 
number of students whp participated in EDY programs: So that the 
figures shown are comparable, the number of students served by 
EDY in fiscal year 1974-75 has been restated to reflect the actual 
number of students who participated m EDY programs. 



TABLE V-10 

Budget Categories of EDY Funds, by Percent'of 
Expenditures, from R^dom Samples of District 
Reports, 1974-7?Through 1976-77 



ERJC 



- these ^patterns. The greatest pervJ^nt of funds was 
spent for classified salaries. TTheT>fcrcent of expen- 
ditures for books, supplies, and equipment replace-* 
"ment was three to four times that^ln ECE, ESEA,^ 
or EDY. ^ ' ; ' \ 

Monies appropriated under the Bihngoal Educa- 
tion Act of 1972 can iM^^sed to employ^ilingual 
eacher aides, purchase>6peciaLbilingual materials 
^and special, equipment, pr^ylae inservice training 
^nd ^taff development, and cover reasonable . 
^expenses, of th|e schBol'i parent advisory group. 
These expenditurSSd-estric^ion^ accounted (bx the 
differences bet>yeen\ the expenditure patterns for 
bilingual ^education i^nds aria/the expenilitufe 
patterns for the pt^jer consolidated 'kpplic^on 
funds. ^ ^ / * ' !l 

\{iller-Unruh 



> 



In 1976-77 a total of $13,849,625 in Miller- 
Unruh^ funds was ai5propriated to ^provide for 
reading specialists. This amount provided for about ' 
1,260^ reading specialists who worked with ah 
estimated li35,66p«^pupils Witltreadin^ needs. 
- — ^nce Miller-^ruh funds ean be use3^orily for 
salahesHor reading specialists, rfb further break- 
down of expenditures is^^ossible. 

iSStheol Plan Review Data'and Program Quality 
Review°Data ^ 

Findings Mated to school plan reviews an^ 
on-site program > quality reviews ^are presented 
b^ow. 




Expenditure 
category 


Percent of total expenditure 


197475 


1975-76 


1976-77 


Classified salaries 
Certificated salaried 


10.0 


17.0 


39.0 /- 




71.0 


62.0 


. 39.0 ^ 




Employee benefits 


IkO 


12t0 


11.0 




Books, supplies, and 
equipment replace- 
ment 


/ * 

(4.0 ^ 




• 7.0 




Capital outlay 




0.5 


, 1.0 




Contracted servives ^ 


2.0 


4.0 \ 


2.0 


v 


Indirect costs 


0 1.0 


0.5 


1.0 




Total 


?(«3.0 


100.0 


100.0 





School Plan Reviews _ 

A total of 3,284 'Elementary school plans were 
submitted to the Department Jn 1976-77. Of these, 
I,013»were updates of existing plans, and 2,271 
were from schools . that were submitting a ^con- 
solidated application for the first time (79^ 
schools) or continuing ECE^^ools that were 
required to submit rewritten program plans^as a 
result of low overall ratings"in M 975-76. Field 
personnel, under the^-tlifection of -Department 
personnel, conducted de'tailed Equality critiques of 
the new or fully rewritten plans. All other plan5 
were reviewed for compliance with applicable legal 
requirements. v/ 

4 For the 1976-77 school 467 secondary 

school plans were received* and rerfd. The plans 
were Yor EDY, bilingual education,^nd ESEA Title 
I programs, including programs for neglected and 
delinquent children in ►high schools, and state 
institutions, such as California/ Youth Authprity 
'centers and state schools for the handicapped. 



■J 
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On-Site Program Quality Reviews ' . 

For 1976-77, program quality review ratings 
were availably for 1,340 elementary schools. Of 
these schools 720 had entered ECE in 1975-76, 

TABLE V-l 1 

Kmnber of Students Served by ESEA Title I and Funds 
Appropriated, 1974-75 Through 1976-77 





Number of 


» 




students served 


Funds 


Year 


(all grade levels) 


appropriated* 


1974-75 


503,400** 


5132,577,018 


1975-76 


'529,800 


130,039,420* 


1976-77 


530,000 


136,827,640 



*AH. figures in this column represent amounts appropriated to local 
educational agencies (LEAs). These amounts do not mclude funds 
allocated for migrant education, handicapped children, neglected 
and delinquent children, or other purposes because the counts oC, 
participants arc limited 'to students receiving funds granted to 
LEAs, ' ' * 

**ln the Evaluation Report of ECEj ESEA Title /, and EDY, 1974- 
75, the number of Title I participants reported (591,561) repre- 
sented a couat of all students in schools that received Title I 
funds. For fiscal years 1975-76 and 1976-77, the summary counts • 
reflect the actual number of students who participated m TifTeS 
programs. So that the figures shown are comparable, the number 
of students served by Title I in fiscal year 1974^75 has been 
restated to reflect the actual number of students who participated 
in Title I programs. 



TABLE V- 12 
Budget Categories of ESEA Title I Funds, by Percent 
of Expenditures, from Random Samples of District 
Reports, 1974-75 Through 1976-77 



Expenditure 
category . 



.Classified salaries 

Certificated 
salaries 

Employee benefits 

, Books, supplies, and 
— equipment replace- 
ment 

Capital outlay 

Contracted services 
Indirect costs 



Total 



Percent of total expenditure 



1974-75 1975-76 1976-77 



43.0 

33.0 
14,0 



5.0 

0.5 

4.0 
0.5 



100.0 



41 

.38.0 
13.0 



5.0 

1.0 

J.p 
LO 



100.0 



43.0 

30.0 
13.0 



8.0 

!0 
4.0 
1.0 



416 had entered ECE in either 1973-74 or 
1974-75, and 204 were receiving only ESEA Title I 
and/or EDY funding. Table V-15 shows average 
program qfqality review scores for elementary 
schools. 

The average scores across all schools reviewed 
were'^^Tcensidered "good" on the basis of the 
Progr^^ Quality Review Instrument scale (0 = no 
evidence; 5 = excellent). The range of scores was 
smaller in 1976-77 than m 1975-76, and the 
average score was slightly higher. Statistics for the 
relationships among the three sections of the 
Program Quality Review Instrument for elemen- 
tary schools are ^given in Appendix H. 

On-Site Compliance Reviews of Secondary 
School Programs 

Monitor and revie^v teams conducted on-site 
compliance reviews at approximately 80 percent of 
the secondary schools that received consolidated 
application funds in 1976-77. In general, funded 
schools are to be monitored at least once every two 
years. However, schools that are.^judged to be 
jsubstantially "out of compliance'* are scheduled 
for a follow-up visit within a year after they receive 
a, noncompliance rating. 

Follow-up monitor and review visits were con- 
ducted in 82 school districts, 164 Title I/EDY 
schools, SO^stitutions ,for neglected and dehn-" 
quent youths and seven nonpublic Schools. 
Another 67 schools were visited for purposes of 
reviewing their bilingual education programs. 

California Assessment Program Reading 
Achievement Data 

Average school readfeg achievement scores from 
the California Assessment Program were used to 
examine the progress of students in schools re- 
ceiving ECE funds. Since all studentsJn a school in 
grades two and three take the California Assess- 
ment Program Reading Tesr 'and ECE funds are 
used to provide services to all students in kinder- 
garten through third grade, the California Assess- 
,ment Program scores can suitably be used for such 
a purptf)se. Grade three scores are used primarily, as 
they reflect the perCormance of-students who have 
completed their period of participation in ECE. Of 



100.0 



ERLC 



course, some students will have participated for 
one year in^'ECE, some for two years, some for 
three years, and some for four years (some whose 
schools entered ECE in 1973-74 will h^ve partici- 
pated '.since kindergarten). Second grade reading 
scores and gain scores from second to third grade^^ 
provide additional juformation about the per- 

41 



35 



formance of students in ECE schools. No single 
analysis can provide a complete picture of student 
achievement patterns in ECE schools. Accordmgly, 
the Department analyzed California Assessment 
Program reading achievement scores from several 
different perspectives, starting from the very 
simple and moving progressively to the ^ore 
complex. The following analyses are presented 

• Historical profiles of grade two and grade 
three reading achievement from 1972-73 
through 1976-77 for ECE schools, by year of 
entry into ECE, as well as for non-ECE 
schools 

* • Longitudinal reading achievement profiles of 

• scjiool scores from grade two to grade three 
for ECE schools, by year of entry into ECE, 
and for a similar group of non-ECE schools 

• Changes in residual readmg achievement 
scores of grade three students m ECE schools, 
by years of participation ui ECE^ 

The reader should note that all tables preseni 
school residual scores in terms of 'weighted 
averages." Weighting is a statistical technique used 
to^ take school size into account m computing 
average scores for groups of schools. For example, 
if student scores for two « schools were being 
averaged ^qd School A- had 150 students tested and 
School B had 200 students tested, the average 
scores (vom Schc5Dl B would be weighted more 
heavily than those from School A to allow for the 
larger number of students tested.. Thus, weighted 
averages for large groups of schools ponray Better 
the average performance of students within those 
schools and|*do not portray as well the performance 
of the average school within the group; 

Hbtorical Profiles of Grade Two and Grade Three 
Reading Achievement in ECE Schools. 1972-73 
Throu^ 1976-77 * 

The first analysis of California .Assessment Pjo- 
gram reading acl>ievement data represents a simple 
approach to assessing \he relationship of ECE to 
grade two and ^de fihree. reading achievement. 
This histoncal approach shows how students in 
ECE schoj;J^erformed over the last five years, 
before ECE^as implemented an^ in each year as 
additional schools were phased into ECE, Separate 
.hjsioncal' profiles are pre:kirtied for all :A:hool5 



within each of the four phases of ECE, on the basis 
of year of entry , and for schools not receiving ECE 
funds. Grade three scores are shown in'TSj^le V-16, 
and grade two scores arfe shown in Table V-17 In 
both tables 1976-77 scores represent the percent of 
questions that students answered correctly on the 
California Assessment Program. Readmg Test 
Scores for the other four years were also taken 
from statewide testing, but ihe\ were adjusted so 
that they are as nearly comparable to the 1976-77 
scores as possible. Obviously, the scores could not 
be made conjpletely comparable because different 
tests were given, dunng the peHod. Statewide 



TABLE 

Number of Students Served bv Bilingual Educatii 
Funds and Funds Appropriated, 
19-^5-^6 and 1976-77 ' 



NumDer of students 




Vear served 


FuDGs aoprbpnated 


]9''5'lb ' 49,100' 


SS, 000 ,000 




•> 


1976-77 46,000" 


8.100.000 



■Estimate from \9'S-^(i Form E-12"'P dau. Of the students served 
26.833 were cUssined as iimited-hngiish speai:ing or non-EngUsh 
speaking: 

•Inaudited estimate from 19^6-"^" Form E-12"P oat a^ Of the 
students sened 28.214 v«ere dassitled as Umited-Engiish speaking 
or non-English sp€:aking: , " 

' TABLE V.I4 

Budget Categories of Bilingual Education Funds, 
by Percent of Expenditures, from Random 
Samples of District Reports, 
1975-76 aiW 1976-77 



Percent or" total expenditure 



Expenditure, categenei^ 



19'75-76 



Classitled salaries 

,^ertiTicated salanes 

^ Employee benefits 

Books, suppbes, and 
equipment replacement 

Capital outlay* 

Contracted services 

Indirect costs** 



5^0 
3.0 
120 

21.0 
0.5 
90 
0 5 



1976-7" 



62 0 
2.0 
15 0 

15.0 
l-O 
5.0. 



A residual achievement score is detlned as the difference 
between a school's actual score (average score of its students) and its 
predicted score. A positive change in residual score indicated 
improvement relative to prediction on the basis of the 4)ackground 
charactenstics 9f the student population. Negative changes in 
restjduals indicated a decbne in this relative performance. 



Total 



100.0- 



100.0 



•Special ^uipme/it used exclusively m bilingual education prV 
grams, 

••Kxpcnditutes in this category are noi permitted b> la*, and 
dutncts arc billed for such expenditures. 
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testing used the* Cooperative Primary Reading Test 
(Coop) in 1972-73, the CAP I in 197^-74, and a 
revised version of the CAP A called CAP H m the 
three subsequent years The unshaded areas on the 
tables indicate scores obtained by schools during 
the years of their participation in ECE. ' 

At least three important inferences can be drawn 
from the grade three data in Table V-16. First, 
students in EQE schools in the aggregate ha^ a 

'^history of low achievement pnor to the time that 
the schools entered ECE. In each year the achieve- 
ment scores of students in the schools that were 
selected to ente^ ECE were low^r than those of 
students in schools not selected for ECE. The 
1^972-73 statewide testing scores demonstrate this 
quite clearly. As a group the schools that were 
selected as ECE Phase I schools had -the lowest 
ac^vement scores. These ECE Phase I schools* 
were followed in order of lowest to highest 
1972-73 Coop scores by ECE Phase H schools^-^ 
ECE Phase III schools. ECE Phase IV schools, and 
non-ECE schools. 

Second, with one marked exception, grade three 
reading achievement scores for students in both 
ECE and non-ECE schools w^re virtually un- 
changed over the five-year penod. The exception 
wasia^CE Phase II schools, in which the percent 
of questions answered correctly increased 1.2 

.percent from 1972-73 to 1976-77. Students in 
ECE* Phase II schools correctly answered' 79.9 
percent of the items on^he Coop test in 1972-73, 
four years later the students in those same schpols 
answered 81.1 percent of the items correctly on 
the California Assessment Program Reading Test 
The scores of students in Phase II schools were 
higher in 1 9^76-77 than those of students innhe 

other types of ECE schools. 




Third, a very slight ihiproMement in scores waS 
noted in ECE schools overall. In pa/t^cular, grade 
three reading achievement scores in schools that 
entered ECE four years ago (Phase I schqols) 
declined during- the schools' first year in ECE. The 
saores improved in the schools' second, third, and 
fourth years in ECE, and by the fourth year they 
were slightly higher than they had been before the 
schools entered 'ECE. The reading "achievement 
scores in Phase III schools .and •Phase IV schook, 
which entered ECE m ^1975-76 and 1976-77,^ 
respectively, improved slightly in their first year of 
participation. Achievement scores in ECE Phase III* 
schools decreased m the schools' second y^ar 
(1976-77) of participation. By contrast, grade 
three reading achievement scores in non-ECE 
schools declined slightly over the five-year period. 

. Table V-17 presents grade two reading achieve- 
ment scores. At least three important mferences 
can be drawn from the data in Table ^ First; 
they show that before ECE schools entered ECE. 
their students scored lower on grade two reading 
achievement tests than "students in non-ECE 
schools scored." 

Second, the grade two scores flijctuated more 
than^ the grade three scores did over the five-year 
pFeriod.* 

Third, grade two scores in ECE schools decline 
slightly during the schools' first year of partici- 
pation in ECE but tended /to improve in^'sub^e- 
quent years, in particular,' the schools that entered ' 
ECE in 1973-74 (Phase I) experienced a drop m 
gr^de two reading achievement; liowever. the^e 
schools have shown some improvement since that 
year. 

The scores in Phase II schools, which entered 
CE in 1974-75, declined during the schools' first 
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TABLE V.I5 

Descripthe Statistics for the 1976-77 Monitor and Review Data for the 49 hems Used for the 
Grades with'EC£ Funding and the 45 Items Used for Grades Without ECE Funding 



Section I 



Development and use 
of the plan 



Number 
of Items 



Item 
mean 



^Grades with 
ECE funding 

Grades without 
ECE funding 



19 



3.6 



Section II 



Program 
implementation 



Number 
of Items 



39 



36 



Item 
mean 

3.7 
3.4' 



Section III 



Ongoing program 
development 



Number 
of Items 



Item 
mean 

3.6 
3.3 



Total 



Number 
of Items 



. 49 



45 



Item 
mean 



37 



37 



. TABLE V.16 

Grade Three CAP Reading Achievement Scores for the Years 197^:73, Through 1976-77 
for Noa-ECE Schools and by Years of Paitippation for ECE Schoob 




*€oop and CAP I were predecessors of CAP II. 



TABLE V.17 



Grade Two CAP ^ding AdiievementScores for the Years 1972-73 Through 1976-77 
' for NodiCE Schools and by Years of Partidpation for ECE'Sdioob 



Type of school 




ECE Pliasc I schools 
(Entered ECE in 1973-74, N = 393) 

ECE Pfiase \\ schools 
(Enleted ECE in 1974-75, N = 624) 



ECE Phase HI sdiools 
♦ (EnterciECE in 1975-76, N = 454) 



ECE Phase-IV schools 
(Entered ECE m 1976-77, N = 6?4), , 



Non^ECE schools 
CN = 1592) 



*Coop and Cap I were predecessors of CAP II, 
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year in ECE but improved during the second and 
, third years to a point well beyond the pre-ECE 
level/The Phase III schools, which enfejed ECE in, 
1975-76, experienced a decline in the first year but 
made 'gains back to the pre-ECE level in the second 
year. The scores in Phase IV schools, which were 
declining prior to the schools' entering ECE, 
declined further in 1976-^77. 
i Examined together, tables V-16 and V-17 show 
an interesting pattern of reading achievement in 
ECE schools from 1972-73 through 1976-77. In 
ECE schools grade two reading achievement scores 
have consistently decreased in the first year of ECE 
panicipation but have increased m subsequent 
*years. Thus, in 1976-77 ECE grad| two reading 
achievement scores were higher |han those in 
1^75-76, with the exception of Ph|se IV schools, 
which entered ECE in 1976-77. By contrast, grade 
three reading achievement scores of students in 
ECE schools have shown less fluctuation, with a 
slight increase in Phase I schools and Phase IV 
schools, a slight decrease in Phase III schools, and a 
substantial increase in Phase II schools. 

The above description of student achievement 
performance must be qualified, ^however, because 
the- schools in the Four phases of ECE and the 
non-ECE schools differ from one another in terms' 



of a variety of student background characteristics. 
Since differences in background characteristics can 
have a. profound impact on student achievement 
scores, the next two analyse;s provide aft oppor- 
tunity to reexamine student achievement scores 
after the scores have been statistically adjusted for 
background factors; that is, they allow for ''con- 
trol" of background factors. The reader should 
note, however, that the adjustments result in a 
certain amount of statistical abstraction in program 
comparisons. The focus of the comparisons is no 
longer simply on test scores; it is on scores^^s they 
appear after adjustments for background' factors^ 
have been made. 

Longitudinal Comparison of ECE Schools and a 
Similar Group of Non-ECE Schoi^ls 

For a- longitudinal comparison of ECE s<:hooIs 
with non-ECE schools, all schools were grouped on^ 
the basis of predicted grade three reading achieve-^ 
ment and additional background factors; and mean 
achievement scores were computed in accordance 
with can accepted statistical procedure for calcu- 
\lating weighted averages. - • 

Longitudinal profiles^-of the four phases of ECE 
schools and of the group of non-ECE schools are 
presentee! in Table V-18. The profiles are longi- 



TABLE V.18 

Reading Achievement Gain Scores for Groups of ECE Schools and Non-ECE Schools, 



Type of school' 


Grade two scores 
1975.76 


« 

Grade three scores 
1976-77 


Gam scores 
(number correct, 
1976-77, less number ^ 
correct, 1975-76) 


Phase 1 schools 
(Four years m ECE) 
N=393 


* 62.5 ' • 


76.5 


-> 

14.0 


Phase II schools 
(Three years m ECE) 
N=624 


67.0 


8L9 


f 

14.9 


Phase III schools 
(Two years m ECE) 
N=454 


* 

65.3 ^ 


79.9 


14.6 


• Phase IV schools 








(One year m ECE) 
N=694 


70.3* 


85.0 


14.7 . 


Non-ECE schools 
N=K992 


68.6 


81.9 


13.3 



•Grade two score i$ prior to entry into ECE 



tudinal in the sense that they trace the change in , 
average school scores frpm grade two. to grade 
three over the span of one year. The first column 
in Table V-18 shows California Assessment Pro- ^ 
gram reading achiejvement scores for*^ade two 
students in 1975-76. The second column shows the 
scores of gradFthfee students in the same group of 
schools on the identical reading achievement tfest in 
1976-77. The third column shows how much* 
progress was made from grade two to grade three. 

The longitudinal* gain scores of students in the 
ECE schools for ea<!h of the four phases were larger 
than those of students in the non-ECE schools, 
which indicates that greater gains were made in the 
^ ECE schools than were made in the n'on-ECE^ 
schools. Several inferences can be drawn ^om th^^^ 
figures in Table V-18. Students in ECE schools- 
with the exception of students in ECE Phase IV 
schools -had 'lower grade two reading achieverhent 
scores in 1975-76 than students in a similar group 
of non-ECE schools had. When grade three stu- 
dents in'the same schools were tested the next year^ 
(1976-77), those in ECE Phase I schools and Phase 
III schools continued to score lower th^h did those 
in the similar group of non-ECE schools. Those in 
ECE Phase II schools and Phase IV schlols scored 
as high as, and higher, than those in 'the similar 
group of non-ECE schools. 



\ 



Changes in StaHtia^ized Residual Reading 
^Achievement Scores of Grade Three Students 
in ECE Schools ^ ' , 

Standardized residuals are a way of reporting 
test scores so that the achievement of students in a 
school can be measured against a relative standard. 
The relative standard is establisheid by a statistical 
procedure called multiple iii^ear regression, in^ 
which a predicted score for each school is com- 
puted on the basis of several school background 
'characteristics. In the analysis presented in this 
section, the background characteristics used to^ 
develop predicted scores were (1), ^he' scores 
obtained by first graders dn the Entry Level Test, 
(2) a determination by teachers of the average 
occupational level of the parents of children inj^the 
school; (3) the number of students tested^^^at each 
grade level; (4) the percent of students who spoke 
a language other than English; and (5) the percent 



from their predicted scores far more than do the 
scores of large schools, an adjustment in the scores 
was made so that the average deviations would be 
about equal for all schools regardless of size. These 
adjuste^^scores are called standardized residuals. 

The criterion. Standardized residuals alone can- 
not be used as a \:riterion (j^?program success. Tliey 
mea.sure achievement levels against the current 
achievement level of students in ''similar** schools,* 
but they do not measure progress. As a -eense- 
quence, the criterion used in these analyses is the 
average change that has taken place in standardized 
residuals in each school since the inception of ECE 
in that school. For the ECE analysis the concept of 
baseline data was introduced. Jn the analyses 
^presented in this section, the criterion value 
the change that had taken place in standardized 
residual scores since the year before the schools 
entered ECE. Thus, for Phase I schools the values 
that are reported in the following tables are the * 
changes that have taken place in standardized 
residuals since 1972-73, the. year before thos^ 
.^schools entered ECE. As a consequence, positive 
values indicate improvement, not absolute success, 
that is, positive values mean that, on the average, 
test performance was higher than it was the year 
before ECE was implemented in those schools. 
Positive values do not necessarily mean that perfor- 
mance in ECE schools has yet surpassed the level- 
of performance in non-ECE schools. 

. ^ The results. Table V-19 shows the ' average 
changes that have taken place in third grade 
standardized residual scores in ECE schools since 
,the year before the schools entered the program. 
On the average virtually- no change (+0.01) 
occurred after one year of participation;*that is, 
the changes in third grade standardized residuals 
between 1973 and 1974 for Phase I schools, 
between 1974 atid 1975 for Phase II' schools. 



TABLE V-19 
Changes in Residual Scores (Weighted Averages) on 
^ Grade Three Reading Achievement Tests After One, 
Two, Three, and Four Years of 
Participation- in ECE 



Number of years in ECE 


Change 


One \ 




+0.01 


Two 




-0.03 


Three 




+0.07 


Four 




+0.03 



of children who had been enrolled in the school tor 
a relatively short period of time. 

The difference between ^ach school's actual test 
5core (average score of its students) and-^ts 
predicted test score is >,Qalled a ^residual score. 
Because the scores of small&^iOols tend to deviate 
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betweeli 1975 and 1976 for Phase III schools, and 
between 1976 and 1977 for Phase IV schools were 
generally almost nonexistent. Similarly, the 
changes after two years of participation in ECE 
(from 1973 to 1975 for Phase I schools, from 1974 
to 1*976 for Phase II schools, and from 1975 to 
1977 for Phase^III schools), were close to zero. The 
only change of any significance was that noted 
after three years of participation in ECE. The 
change after^ four years is also practically non- 
existent. (Only Phase I schools have been in ECE 
for four years; thus, the value of +0.03 is the 
average change that has taJcen place in s'tandardif^ 
residuals from 1972-73 to 1976-77 in Phjse I 
schookonly.) ' J ^ \ 

Tal>l^.20 'shows a more detailed breakdown of 
the changes that have taken place in standardized 
- residflals. In this table the changes are shown for 
subsets of ECE schools. Tile subsets were estab- 
lished by dividing the schools ^on the basis of the 
Entry Level Test scores fo^ those schools in the fall 
of 1973 (the first year in which ll^e test was 
ftlministered). Thus, the results could b^eviewetl 
for three categories of schools: (1) those that 
began with Students who,^n the average, had a low 
level of readiness skills upon entering school; "(2) 
those whose students^ were slightly below - the 
statewide average in readiness skills; and (3) those 
whose students were, on the average, above the 
statewide average in readiness skills. 

The overall ave^rage results that show essentially 
no change do nof hold true for each subset of 
schools." In fact, the average of no change is a 



TABLE V-20 
^Changes in Residual Scores (Weighted Averages) 

on Grade Three Reading Achievement Tests 
% After One, Two, Three, and Four Years. ' 
of Participation in ECE, by Three 
Levels of Performance on the 
1973-74 Entry Level Test 



Number of 
years in 
ECE 


Changes in residual scores^by 
* ^ percentile rank, on the 
1 973-74 Entry Level Test 


Low , 
(1-20) 


Middle 
(21-60) 


• High 
(61-99) 


One 


+0.04 


-0.04 


+0.05 




-0.11 


-0.03 


.+0.05 


* Three 


^.02 


^+0.11 


^ +0.11 


' Four 


-0.08^ 
— » — *' 


+0.18 


-0.01 



combination of three totally different patterns for 
the three subsets of schools. The schools that had 
low Entry Level Test scores in 1973-74 have shown , 
a pattern of slightly increased residuals after one 
year^^of participation but decreasing test scores 
from that point. XMuch of the decrease that drew 
careful scrutiny in last .year's repovt has disap- 
peared.) The scores of the schools in the middle 
group, on the other hand, have, on the average, 
decreased slightly 1 after the schools' first and 
/Second years of participation in ECE. This decrease 
' has been followed by substahtial increases after the 
third and fourth years.. The grouji^of schools with 
the highest 1973-74 Entry Level Test results had 
shown strong gains in scores ufrtiLJ976-77, A large 
decline in scores for the Phase I schools in 1976-77 
negated the gainsr reported in 1975-76, 

Another way of rev^^ing the , test results is to 
examine the total change ^in standardized residuals 
since the year beforejhe ECE schools erjtered^ECE 
(the baseline year).\niese results are^displayed in 
Tajble V-2L Again tKe results show little change 
when averaged across all schools. Pha/e^I schools 
have made the greatest gains, with standardized 
residuals up 0.09 since/^ their baseline year^" 
(1973-J4); Phase III schjioas are the only set with 
residuals this year that are lower than they were 
the year before entering ECE. This decline is 
primarily the result of a sharp decline in stan- 
dardized residuals by schools in the middle Entry 
Level Test category. 

These same results can be shown another way. 
Table V-22 shows the number of schools (and 
number of pupils in those schools in parentheses) 
that since their baseline year, have had increasing • 
residuals, decreasing residuals, and unchanged 
residuals. Unchanged residuals are defined as those 
that have changed by less than .0.5. Increased 
residuals ,are th^ that have increased by 0.5 or 
more, and decreaseWesiduals are defined inversely. 

The results in Table V-22 are basically a 
repetition of those in Table V-21. For example, 
^one can see frpm Table V.21 that the declines for 
Phase III schopls in the middle Entry Level Test 
category were a result of a decline in residuals in 
74 schools containing 4,108 third'grade pupils and 
that residuals have risen .in only 42 schools 
containing 2,421 third grade pupils. -Conversely, 
i'hase i/middle Entry Level Test schopl^ had the 
gfeate^ increase in standardized residtfals shown in 
Table V-21 ,(+0.18). This result is verified by the 
data in Table V-22; 67 of these schools had- 
increasing residuals, while only 43 had decreasing 
residuals. 



ERiC 



< 



41 



Nonetheless, interesting discrepancies pan be 
noted. For example, while the average change in 
residuals was positive for Phase 11 schools. Table 
V-22 shows that, in fact, a few more schools have 
had declining residuals than have had increasing 
re^duals. Thus, the average positive gain is the 
result of two factors. (1) the schools that gained, 
are larger than those that declined (this is shown 
by the fact that while more schools have been 
declining than gaining, the total number of stu- 
dents in the gaining schools is larger)', ancl (2) the 
g^ins in the inci^easing schools are greater than the 
losses in the declininjj^chools. ' ' 

Table V-23 portrays the, changes in standardized 
residuals in stiU another way. The table is a 



replication of Table V-20, but with the comparable 
scores from the 1975-76 evaluation report added in 
-—parentheses. In 1975-76 the values reported in the 
row for three years of ECE participation were 
those of the Phase I schools, which had then just 
finished their third year in ECE. Those same 
schools have now completed their fourth year of 
participation, so it is reasonable to compare their 
changes in residuals from the baseline data after 
their fourth year to last year's changes after three 
years. Similarly, the results for two years and three " 
years of participation are comparable to last year's 
results after one and two years of participaticm. 

These result^show a substantial decline in scores 
across the* board from the first to the second year 



TABLE V-21 

Gianges in Third. Grade Standardized Reading Residuals from 
Baseline Year Through 1976-77 









Phase (year of entry into ECE) 


Entry Level Test 
percentile range 


1 

(1973-74) ^ 


. (1074-75) 


111 

(1975-76). 


(1976-77) 


Low (1-20) 
Middle (21-60) 
High (61-99) 


-0.08 
+0.18 . 
-0.01 


+0.16 
. +0^08^ 
+0.07 


-0.02 ^ 
-0.20 * 
+0.03 


+0.13 ' 
•-0.04 
+0.06 ' 


TcJtal 


* +0.02 * 


+0.09 


^ -0.07 


' +0.04 , 



FABLE V-22 



Nurnbers of Schools (and Pii^iBprhat HaVe Had Decrea^^g, Unchanged, and 



01 ^n^ls (anc 



ird Grade Standardized Reading Residuals 
Baseline Year Through 1976-7^ 



Phiase 



Entry Level Test percentile range 


1 


II ' 


III 


IV 




Decreasing 


48 (3,941) 


56 (3,223) 


41 (3,029)" 


40 (3,156) 


Low 


Unchanged 


30 <h901) 


35 (2,388)^ 


26 (1,941) 


47 (3,844), 


.(1-20) 


Increasing 


•I 47 (3,454)_ 


57 (3,856) 


37 (2,649) ' 


^ 50 (3,771) 




Decreasing 


/ 43 (2,5^) 


•88 (5,040) . 


74 (4,108) 


- . 86 (5314) 


Middle 

* 


Unchanged 


47 (2,598), 


87 (4,810) 


62 (3,170) 


106 (6,159) 


(21-60) 


Increasing 


67 (3,700) 


S7 (5,5l34) 


42 (2,421) 


' 74 (4,628) 




Decreasing 


37 (2,010) 


57 (3,019) 


46 (2,491) 


^65 (4,041) 


High 


r Unchanged 


35 '(2,083) 


92 (4,984) 


76 (4,113) 


130.(7,810) 


. (61-99)^ 


Increased - 


•30 (1,436) 


54 (2,868) 


47' (2,413) 


. 87 (5,M24) 



r 



id 
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of ECE participation.' Conversely, even stronger 
^ gains are shown across the board from the second 
to the third year. The changes in Phase I schools 
fronx last'yeai- show distinctly different patterns 
.for the three subsets of schoo^. On the average, 
schools in the lowest Entry Level Test grbup had 
extremely large drops in standardized • residual 
scores after three years of participation. This year, 
while their scores are still lower^than they were 
before entering ECE, they are substantially] in- 
creased over last year. Schools in the middle E^try 
Level Test group showed strong gains last year; 
those gains have been sustained. On the other 
hand, schools in the highest Entry Level Test 
group, which last year had scores that were ilmost 
as- high as those in the middle Entry Level Test 
^ group, lost all of their gains and currently are 
scoring, after four years in ECE, almosTexactly as 
they wke before they began participation in ECE. 
, Table V-24 is the most detailed presentation of 
all. The changes in residual scores from the baseline 
year are shown for each phase of ECE schools after 
each year of participation. >^^,^ 

Publishers' JNJorm-Referericed Test Data 

* Fgjr each type of funded school program, three 
types of analyses are conducted using the standard 

TABLE V-23* 
Changes in Residual Scores (Weighted Averages) oh 
Grade Three Reading Achievement Tests After 
One, Two, Three, and Four Years of 
Participation in ECE, by Three Levels of ' 
Performance on the 19^73-74 
Entry Level Test 



Number of 
years in 
ECE 


Changes in residual scores, 
grouped by percentile 
.rank on the, 1973-74 
Entry Level Test 


All ECE 
schools, 
(weighted 
average) 


1-20 


21-60 


61-99 


^One 


+0.04 


-0.0,4 


+0.05 


+0.01 


Two 


-0.11 


^0.03 


+0.05 


-0.03 


7 


^ (0.00) 


(+0.05) 


(+0.06) 


(0.00) 


Three 


-0.02 


+0.11 


* +0.11 


" +0.07 




(-0.15) 


' (+0.05) 


(+0.06) 


(-0.01) 


Fclir 


-^0.08 


+0.18 


-0.01 


+0.03 




(-0.24) 


(+0.19) 


(+0.18) 


^ (+0.03) 



NOTE Changes m residuaJ scores reported on page 36, Table V.9 
of Evaluation Report of ECE, ESEA Title /, and EDY, 
19 '^5 "6 for the same groups of schools are given m 
parentheses. 



scores, from publishers' norm-referenced -tests 'in 
reading and mathematics. The\ analysis of most 
interest is that in which a comparison is made 
between the position of students relative to the 
norm group, by type of program (for efxample, 
students in all schools with., consolidated appli- 
cation programs), ancj the growth of students in 
the publishers' norm group.- ^ ^^econd ^analysis 
involves an examination of the progress of the 
students from the time of'the pretest to the 
post-test, relative to the publishers' norm group. A 
final analysis'involves the progress of students, by 
grade level, within each of the above analyses. 

All Schools with Consolidated Application Funding 

Figure V-3 is a summary of student achievement 
on standardized reading tests for all schools that 
reported scores from^a publisher's norm-referenced 
test. The scores^ only those students who were 
classified as fluent-English speaking and who were 
tested at the correct . level of the publisher's 
norm-referenced test for their age and grade ^are 
represented in the figures. Summaries of achieve- 
•ment scores for students who^were tested out-of- 
level appear in Apjjendix I. 

Reading. The reading achievement data in Figure 
V-3 indicate* that test^core gains of program 
participants exceeded norm group progress at all 
grade levels, with the exception of grade eleven. 
The standard scores in the eleventh grade firopped 
from. 40.8 on the pretest to 40.7 on the post-test. 
In terms X)f raw score points (not s^own), eleventh 
grade students made progress between the pretest 
and the post-test, but their rate of progress was less 
"than that of the norm group. Thus, their standard 
score declined by O.I during tHfe year. 
, The data for grades one to three are for the 
greatest number -of participating schools and stu- 
dents. 

The reading achievement data in Figure V-3 
indicate that student participants at all grade levels 
were scoring below the national average of 50 
standard score points 6n the pretest. At all grade 
<^ levels excepj grade three, student participants were 
still scoring" below the national average of 50 on 
the post-test. In grade three, students advanced 
from 46\9 on the protest (3.1 below the national 
average) to 50.2 (0.2 a&oveHth^ average) on the " 
post-test, thereby showing a gain of 3.3 standard 
•score points and registering greater growth th^ ' 
students at any of the Other grade levels. 

Mathematics. Figure V-4 preseitts a summary of 
student achievement on stahdarfiized mathematics 
tests for all participating students. At all grade 
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levels student partfcipanls ^scored * below the 
national average on the pretest. Students in grades 
one and two scored above the natipnal average on 
the post'test, but those in all other grades scored 
below the national average. Despite their lower 
absolute performance on the pretests and post- 
. tests, students 'showed more growth than the 
publishers' norm group at all gtade levels except 
grade eleven. The greatest growth was shown in 
grades one, two, and three, which include the 
'largest' number of participating scho9ls and stu- 
dents. The largest gain (5.1 standard score pomts) 
jn mathematics was made by students in the first 
grade. ^ * 

Multiple-Funded Schools • ' 

This section contains mfoririation about student 
achievement m schools receiving funds from more 
than one funding source. The nciajority of these 
schodls received a combination of ECE, ESEA' 
Title I, and EDY funding, but schools that received 
bilingual education and MillerUnruh funds are also, 
included. Reading and mathepiatics scores for' 
students participating in the multiple-funded 
school programs are shown in Figure V-5. - 

Reading, Analyses of' weighted standard scpre 
gains, revealed that across all grades, st^ents in 
multiple-funded schools Typically gained l\2 stan- 
dard score points. Thus, across grades, students . 
made greater gains than the publishers' norm m^up 
from pretest to post-test* A compjarison bet\^en- 
pretest and post-test standard scores in leading for ' 
participating students in school '^rbgra^^s sup- 



ported by more than^ one funding source showed 
that third grade participants -typicayj^made the 
greatest gains. The third -grade sc^^T increased 



froln 45.rjeJo 48.4, a difference of 3^.3 standard 



score pQints. The finding^ displayed in>Figure V-5 
indicate Jhat students in grades one through eight 
.(except for grade seven) made the greatest gains 
and that students in graces nine through twelve 
tended to have progressively lower. priest- and 
post -test scores. ' • / ^ 

Mathematics. Participating students in school 
programs supported by more than one funding 
sol!ll^ increased in positiofi re|^|^e to thed^^nal 
average at all grade levels except gra^e elfven. 
Students in the primary grades, made the greatest 
gains. The pretest and post-test standard scores in 
mathematics shown in figure V-5 indicate that the ^ 
greatest gains «were made in grade one, where scores 
increased 5.1 standard score points from 45.9 on 
^the.pretest to 51.0 on the post-test. Students in 
gradeis seven and higher showed a slight gain 
relative to the publishers' norms. Students in grade 
eleven were an exception, again, they showed less 
progres^^lative to the norms. , 

AU ECE Schools 

Since 1972-73 ECE has provicjejd a framevvork 
Xor*the restructuring of tli^e educational process in 

landergart<?ir^tI^ough grade three. ECE support has 
^enabled schools to integrate both b'asic und cate- 
'/goriCMiffesources to meet the needs, of students 

reqUi^P different types of services. The restruc- 
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Changes in Residual Scores (Weighted Averages) on Gr^de Three Reading Achievememt Tests After, 
One, Two, Three, and Four Yews of Pa^icipatian in ECp, Year of Entry into 
ECE, for TJbree Levels of Peitbrmance on the 1973-74 Entry Level Test ^ 
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3. Pretest a^id post-test standard scores in reading achievement|1^.grade level, for students partidpat- 
ihg in school programs supported by consolidated funding, 1976-77 
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Fig. V-5. Pretest and^^ost^test standard scores in reading and matnematics^chievement, by grade' level, for 
students participating in school programs supported by more ti&ri^one funding source, 1976-77 
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turing involved the total school environment in the 
primary grades. ^ / 

The findings reported in the Hrst part of this 
section* are for all schools engaged in restructuring^ * 
(all ECE schools) regardless of whether they 
received other types of aid. 
, Pretest and post-test standard scores for students 
participating ;n ECE w,ere used to (1) measure 
reading and mathematics achieve nji^nt gains; and , 
(2) determine the relationship betwe^n the length 
of participation in ECE and changes in ^jjfeent 
performance in reading^ and mathematics.' 

Reading. Figure V-6 shows reading achievWient 
scores for students in all schools that recakgd^ECE 
funds. On the average, students in EC^Ichools 
were progressing in reading at 'a rate greater than 
that reflected by publishers' norms. Pretest 
achievement scores in grades one, two, and three ' 
wQre from 0.7 to 2.2 points below the national 
norms^^^ut post-test scores ranged , fro^ti. the 
national norm to 1.1 points above the national 
norm. y 

Mathematics. Students in schooU that received 
ECE> funds showed substantially "greater growth^ 
than the publishers' norm groi^ showed. The^ 
greatest growth was ill grade one. As shown in 
Figure V-7, scores increased from 47.6 to 52.6, a 
difference of 5.0 standard score points. The 
average post«-test achievement for students in all 
ECE schools was ajbove the i^tional average. - 



Figures and V-9 show average 1976-77 
reading'^and mathematics achievement 'scores for 

^tudems in all ECE schools, grouped by the year in 
which ^the schpols ^nte'red ECE. Qverall, student 
performance improved from the pretest to the 
post-test at a-rat& in excess of the publishers' 

^ norms regardless of when schools entered ECE. 

Schools^at Recced ECE Funds and Compensatory" 
Education Funds 

Figure V-10 contains the reading an1i mathe- 
matics aehievemen^ scores on ^ublishcH^norm- 
r^ferenced tests for students in gfade«s^e, two, 
and three who were in ECE schools inat also 
received compensatory education funds (ESEA 
Title I and/or EDY funds). ' 

Reading, In schools that received ECE funds and 
compensatory education funds, students pro- 
gressed at a rate greater than that of the publishers' 
norm group at \\\ ^ade levels. Gains^elative'to the 
publishers' norm group were progressively greater 

Jrom grade one to grade three, with students at 
grade three making the greatest gains j[3. 4 standard 

'Iscore points).* Pretest scores were progressively 
lower from grade one to grade three. 

J Mathematics, In mathematics <he greatest gains 
^between the pretest and post*test \^ere made by 
students in grade one. Their ^cores increased from 
45.9 to 51.2, a difference of 5.3 standard score 
points. In grades two and three, students gained 
3.2 and 3.3 standard score points, respectively. 
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Fig. 'V«6. Pretest and post -fast st^dard scores in reading achievement, grade leve^^, for all scliools partici- 
' pating in early childhood education funded programs, 1976-77 
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Schools That Received Only ECE Funds 

The jchools reported on in this section received 
,only E(^E funds, the pretest scores in ECE-only 
schools were higher than those,, in schools that 
received BCE funds and compensatory education 
(EDY/ESEA Title I) funds. 

Reading. Figyre V-11 shows average student 
reading achievement scores for students 'in grades 
one, two, and three in schools that received only 
^CE funds. The avexage pretest score for grade 
thr^ students in ECE-only schools was 50.4 
standard score points. The average post-test score 
in suciflfciools was 53.8, a gain of 3.4 {joints more 
than would otherwise have been expected in* one 
year of^ijistruction^ While students in ECE-onl^ 
schools, typically scored close to or at the norm on 

ft the pretesf, they exceeded the publishers' norms 
by 1.7 to 3.8 points at each^gradp level on the 
post-test. This result suggest&^Jthat students in 

^ ECE-only schools were gaining relative to pub- 
lishers' norms on norm-referenced reading tests. 
Similar finding were reported in the co*nsolidated 
evaluation reports for 1974-75 and 1975-76. On 
the basis of 1974-75 data, attempts were made to 
limit the possibility of biased norms being respons- 
ible for such significant gains. The statistical 
procedures that were employed to mitigate this 
bias reduced the gains only slightly and confirmed 
the finding of significant gains from pretest to 
post-test. A full discussion of within-year mea- ' 
suring. of achievement gains is presented in the 
methodology section of this report. 



^Mathematics. The data in Figure V-1 1 show that 
students in ECE-only schools scored* from 2.7 to 
4.3 standard score points above the publishers' 
norms on the mathematics post-tests. 

All SchooIrThat Received Compensatory 
Education 'Funds 

Some schoojs received compensatory education . 
funds (ESEA Title I and/or EDY funds) to provide 
services to students with special educational needs. 
Many of these schools also received other funds, 
such as ECE, bilingual education, ^dMUler-Unruh 
funds. V 

^ The achifevement scores presented in this section 
are for fluent-English-speaking students. The data 
for limited-English-speaking students are disclissed 
in a separate section. 

Figure V-1 2 contains achievement data in read- 
ing and mathematics for students in school pro- 
grams supported by ESEA Title. I and/pr EDY 
funds either solely or in combination with funds 
from one or more of the other funding sources. 

Reading, Students in schools that received com- 
pensatory edi^tion funds progressed at a rate 
greater thm that of the publishers' norm group at 
all grade levels. The largest gains in reading were m 
grades one through isix, with the gains in grade 
three being the greatest. In grade three average 
student scores increased from 44.2 to 47 J, a gam 
oj^ 3.5 standard score points. 

Post-test scpres were below the national average 
at all grade levels. Post-test scores in grades one and 
three were closest to the national average. 
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Fig. V'T. Pretest and post-test standard scores in mathematics acjiievement, by.grade level, for schools" 
participating in early childhood education funded programs, 197d-77 * ' . . 
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Fig. V-10. Pretest and post-test standard scores in reading and mathenratics achievement, by grade fevel, for 
Students in »±ool programs supported by 'ECE and compensatory education funds, 1976-77 
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Fig V-l 1 Pretest and post-test standard scores in reading and mathematics achievement, by grade leveL 
^or schools that receiypd only ECE funds, 1976-77 ^ 
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^ Fig. V-12. Pretest and post-test standard scores in reading and 'mathemati(?s achievement, by grade leve!^f< 

all schools that received compensatory education funds, 1976-77 ^ 
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Mathemmics The gain scores in mathemati<;s for 
stu^ients fei ssjiiools that received compensa^bry 
educa'tion funds were greater thaj^ those of pub- 
lishers' norm groups at all grades except grade 
eleven. Post-test scores were below the national 
average Students in grades one through four i^ade 
^the greatest gains in mathematics. More .specifi- 

" tally, tlie greatest gains were made in grade one, 
with the average score increasing from 45.2 on the 

, pretest to 50.6 on the post-test, a dif^ence of 5.4 
standard score points. The amount of student gains 
decreased froni elemenfary schools to secondary 
schools. Students in^grade eleven showed a slight 
loss, fhe average 'score in inathemati9S in'^ grade 
eleven dropped from 42.2 to 42.0. 

^ ' } • 

,Schook That Received Only E§EA Title I Funds 

V This section contains information about student 
a\hieyemen^ in school programs supported by 
E^A Title I funds only. Data for schools with 
Titl^ I funds to serve handicapped and neglected 
anci^delinquent youth. are provided in a separate 
section. ' ' ^ 

Figure V-is zohi?^U^Mlti^ox grades one 
throu^ , twelve. The rea(ler<.^ould note that 
relatively few secopdapy 5Qh4>ls conducted, pro- 
grams, and that out-oMfev^ testing* was conducted 
\TLj(^?m^^J^^'^ri schools. A comparison of- 
l976-7*rs£ores with those from 1975-76 showed 
that both^the pretest *and post-t'CSt scores declined 
slightly in scljools that reaeive.d pnly ESEA Title I 
funds. These declines may be the result of two 
factors: ( 1 ) more schools were testing on an annual 
(spring/spring) basis, in which case estimates of 
gains are morp conservative, and (2) imprpved data 
collection procedures provided for ^he ehmination 
of non participant scores from the data pool. It has 
been determined that for previous reports, a few 
districts submitted d^a f^r alfstudents rather than 
just program participants at tW secondary level. 

Figure V-IS shows reading and mathematics 
achievement scores for students in programs sup- 
ported only by ESEA Title I funds. 

Redding^ Across all grades participating students 
in schools that received only ESEA Title I funds 
typically* gained 2.4 standard score points from 
pretest to post-test. Their rate* of progress was 
greater than that of the publishers' norm group. 
Par ticipati ng students in grades one through three 
made the greatest gains from pretest to post-test. 
Paj[1icipating students in grades seven through 
twelve tended to have'^progressivejy lower gain 
scores. . ' ' 



i\kithematics. Across all grades participating stu- 
dents in schools that- received pnly ESEA Title I 
funds typically gained 3.3 standard score points in 
mathematics achievement. Although bo^h pretest 
and post-test scores were below the national 
average at all grade levels, gains were greater than 
thos(^ of the publishers' norm group at every grade 
except gradp eleven. Participating students in grade 
one and grade three gained 7.9 and 7.7 standard 
score points^ respectively. 

ESEA Title I Programs for Handicapped Students 
and Neglected and Delinquent Youth 

During 1976-77 ESEA Title I funds were 
provided to ^ serve 35,946 students who qualified 
for ' services^'in special compensatory education 
programs. These programs were provided uhder'the 
auspices of the Departments of Education, Correc- 
tions, and Healtfi; CaUfornia Youth Authority; 
offices of county superintendents of schools; and 
school districts. 

Services were provided for 5,146 handicapped 
students in special schools bperated by the Depart- 
^ment of Education and in state hospitals operated 
by the Department of Health; for 3,886 delin- 
quents in instituti6ns operated by the California 
' youth Aiithority; and for 314 felons in institutions 
, operated by the Department of Corrections. Local 
^ educational agencies provided-services for 26,888 
neglected and deUnquent youth. The number of 
participants, by age and approximate grade span, in 
special compensatory, education programs sup- 
ported by Title I funds -is presented in TiWe 
V-25. ' 

Programs for handicapped students in special 
state schools. Programs at special state schools 
served student^ when local educational agencies 
were unable to meet the students' specific educa- 
tional needs. The services provided at these schools 
included comprehensive diagnostic evaluation and 
counsehng services for the parents and families of 
handicapped students. 'The schbols also partici- 
pated in professional internship programs and 
teacher training programs with the' University of 
California and the California State University and 
Colleges. 

Six special state schools administered by the 
Department of Education received ESEA Title I 
funds during 1976-77 to augment instructional 
— programs- for ^]ie__ne^QlQgically->.handicapp^<| . 
blind, and deaf! A total of 1,169 handicapped 
students in special schools participated in such 
instructional pjograms. Of that number, 100, or 9 
percent, were rieurologically handicapped; 121, or 
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10 percent, were blind; and 948, or 81 percent, 
were deaf. Ptograms were in operation between 
184 and 206 school days. Participants in the 
schools for neurologically handicapped students 
attended plasses between three and nine months. 
Students in schools for the deaf and the blind 
attended classes for the full^school year. ^ 

The Department of Health operated programs in 
1 5 mental health facilities located throughout the 
state. These programs operated between 200 days 
and 36.5 days. The average period of attendance for 
the developmentally disabled was from ^ to 12 
months; for the menta lly d isabled it was from, ^ to 
8 oionths^ 4@9^ > 

The ESEA Title I aHocatio'ns to the Depaijjment 
of Health provided/or the establishment of ^p^- 
mentary educational components in state ^nd local 
health treatment programs. A total of 3,«18 
handicapped studer^ts participated in these activi- 
ties. Of the student participants 2,263, or 59 
percent, Were developmentally disabled, those 
whose special needs were, the result of emotional 
stress, psychosis, drug abuse, or the like; and 
1,^3, or 34 percent, were mentally disabled. 



Compounding these functional limitations, for 
many students were secondary handicaps affecting 
vision, hearing, ambulation, and metabolism. Be- 
cause of the severe nature of their emotional 
disorders, the students' ieaming difficulties • cen- 
tered around reading and listening, comprehension 
of words and symbols, and writing and speaking. 

The Neuropsychiatric Institute at the University 
of California, Los Angeles, received ESEA Title I 
monies to SQrve students* with js&v^ handicaps. 
'During 1975-76 a totanof^3 fecial needs 
students received augmented mstruciional services 
, in this multidisciplinary* hospital setting. All stu- 
dents were admitted to tHe institute on the basis-of 
medical Referrals and accompanying problen\s of 
personal adjustment. The program was designed to 
meet the unique needs of the students in terms of 
both their emotional needs and their-^^^cademic 
abilities. [ ^ 7^ 

Programs for neglected and delinquent youth, 
ESEA Title I programs served neglected and/or 
, delinquent students in a variety of special institu- 
\tions. The programs included those administered 
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by local ecjucational agencies, thS California Youth 
Authority, and the Department of Corrections.^ 
While complying with conditions necessarily 
imposed by the institutions, each agency or. insti- 
tution *was required^ fo develop a comprehensive 
educational' plan ^ its use of ESEA Title I funds, 
this plan included both instructional and 
instructional-support services for the students . 
served. ^ • • ^ 

The ppmary objectives of nnost pro-ams for 
neglept^ and delinquent youth were to raise 
acadehHc achievement and to promote attitudinal 
changes toward themselves, their peers, 'and society 
as a whole ^ To achieve these objectives, staffs in 
many of ^e institutions concentrated on coun- 
seling and on a diagnostic/prescriptive instructional 
approach related to individual student needs. 

Because of the short period of time spent in the 
program by a majority of the participants, the. 
interval between pretesting *and post-testing was 
not great enough to provident for a meaningful 
interpretation of norm-referenced test resultS. In 
addition,. the pretest and post-test results reported 
for those participants*Avho had been five month*s or 
mbre in the program represent such a minute 
percentage of those being Served that any con-* 
elusions regarding improved performance would be 
tentative at best! Improvement was reported in the 
afeas of social and communication skills. 

Programs administered by the California Youth 
Authority and the California Department of ^ 

'^Corrections. ESEA /Title I funds are allocated ekch 
year for qualifying students consigned to 'the 
CaUfomia Youth Authority (CYA) . from both 
juvenile and criminal courts and for those com- 
mitted to the Department of Corrections * from 

* criminal courts. 

All students within these institutions are eligible 
for service, but because of financial constraints, 
only those persons identified as "most in need of 
remedial instruction in reading and mathematics 
are selected for participation in the program. 

Programs administered by a local educational 
' agenc}\ During the 1976^77 school year, 109,977 
neglected and delinquent youth received educa- 
tional services provi(|ed^by 170 public and private 
institution^administered at the^ school districjjTor 
county level. Of this number, 26,188 students 



received additional educational services provided 
under ESEA Title I. The number of students 
*served- listed by type of institution, age span, and 
approximate ^ade level -is presented in Table 
V-26. 



Of, the total number of students served under 
ESEA Title 1 programs administered by local 
educational agencies, 77J (about 3 percent) were 
classified as being either limited-. or^ non-English- 
speaking students, while 1,349 (about 1 percent) 
of the total neglected and delinquent population . 
wera classified as being limited-; oc norrcnglish 
speaking. ^ *• 

T^e avj^rafee length of participation m programs 
supported by ESEA Title 1 funds ranged from less 
than three months for 73 percent'of the atudente 
to more than six months for-15 percent ofjth'e 
students. \ 

During 19^6-77, services were provided to 4,200 ^ 
students irtien, institutions operated by the Califor- 
nia Youth Autl^ority and three institutions operated 
by the Department of Corrections. 

The emphasis in ESEA Title I programsjn these 
institutions wa^ on diagnostic/prescriptive instruc- 
tion in reading, Janguage,«and mathematics. 

Schools That ReteivedpnIy EDY FundT^ 

This section contains findings about the reading 
^,and mathematics ' dthievement of students whp 
participated in EDY programs ill schools in which 
EDY monies - were the only categoricjal funds 
received. Very few schools received only EDY 
' funds. ' . . ... 

Figure V-14 ,shows reading and mathematics 
achievement for students in jprograms supported 
o'nly by EDY funds. - 

Reading. Across all grade levels participating 
students in sshools tha^eceived only EDY funds . 
gained 1.9 standard score points 'fro^a^he pretest 
to the post-test, they progressed „«ria rate greater 
than the publishers' norm group. Students 
grades one, through seven appeared to have made* 
the greatest progress relative to the publishers'^norm 
group. "Too few schools reported c^ata in grades 
nine ^through twelve for a reliable interpretation of 
tile differences from the pretest to the4post:te"lst. 

Mathematics. Staridard scores in mathematics 
achievement are shown in Figure V-14 for partici- 
pating students in grades one through eleven in 
school programs supported solely by EDY fundsg, 
No scores were available for gra4e twelve students. 
Post-test standard scores were higher than pretest 
scorfes for all grade levels except grade. eleven. An 
analysis of the weighted average of standard score , 
gains ^across grades;* revealed that participating 
students in ED Y-only^ schools gained 2.4 standard 
score points. 
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Fig. V-I4. Pretest and post^est staftdard scores in readitjg and mathematics achievement, by. grade level, for 
students in school programs supported by EDY funds only • 
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AU Schook with Programs for Limited- 
and Non-EngUsh^peaking Students 

This seQtion of the, report includes information 
about th^ following: 

Reading and mathematics achievement of 
limited-English-speaking students who partici- 
pated in school program? supported by con- 
solidated funding sources ' 
• Reading and mathematics achievement of 
. liinited-English-speaking students wnQoartici- 
pated in school programs supported byinlin- 
gual education funds only 

•All scho6ls that receive ECE, ESEA Title I, 
EDY, or bilingual education funding are required 
to address the needs of limited- and non-English- 
speaking students in preparing their comprehensive 
program plans."* The bilingual education^ com- 
ponent of the school program is 'to include 
fluent-English-speaking students and limited- and 
non-English-speaking students * as pr6gram 
participants. 

-z ^ ' . 

^Vhether or not it receives consolidated funding, each school 
with an enrollment of ten or more hmited- or non-English-speaking, 
students at a particular grade level is required to provide an 
instructional program to address the , educational needs of these 
stu dents. Smcc nonfunded schools are not required to submit Form 
E-127P evaluation data, however, siich schools are not represented 
m this report. * 



Figures V-15^and V-16 show pretest and post- 
''test standard scores in readirifsand mathematics, 
achievement, respectively, forMitrtited-English- 
speaking students in school prog^^p^upported by 
consolidated funding sources. Limited-English- 
speaking students were determined to have limited 
English-language fluency at tHe time of the pretest; 
Accordingly, tkese scores for limited-English-" 
speaking students would not be the soje index of 
the§e students', cognitive skills. Their scores^n 
reading and ^mathematics tests written in EngHsh 
may be lower than average because of the diffi- 
culty they have reading die test questions. . ^ 

No ddta are presented below for non-English- 
speaking students. Until suitable measuring instru- 
ments are available in the primary language of 
non-English-speaking students, attempts to mea- 
sure the acl^ievement of non-English-speaking stu- 
dents in basic skills would be inappropriate. 



Reading. Figure V-^ shows that limited- 
English-speaking students scoreij .below the 
national average in reading at all grade levels. As 
stated previously, however, one can expect the 
pretest 'scpres of limited-English-speaking students 
to be substantially below the jjational average 
because of the^tudents' limited language ^fluency 
at the time of the pretest. 



TAB^V-26 

Neglected an^pelinquent Students in Local Educational Agency Programs 
Served by ESEA Title 1, 1976-77 



Type ol 
program 



hhimber of students served, by*grade level and-age of students 



Prcschaol and 
kindergarten 
2 5 years 



Hlemcnlary ^ 
6- 1 1 yeats 



Junior high 
* school 
1 2*- 14 years 



Higli school 
'>I5-1^ years 



Adult 
19-21 years 



Total 



Delinquents in 
court schools 

Delinquents notC 
in court schools 

Programs serving 
both neglect(id * 
and dehnquent 

Programs serving 
only ru.'glcctcd 

Tuional* 



0 
0 



353 



717 



1,911 



2,886 



4,108 



8,071 



314 



14 
0 



445 



2,292 
618 



, 3,055 
950 



0 
0 



Total 



14 



2,666 



7,707 



16,184 



317 



6,686 
11,677 



6,512 
2,013 



26,888 



* hilorul jvsKtancc fur both ncjdcctcd jnd delinquent students in addition tu their regular school program 
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Fig. V-15. Pretest and post-test standard scores in reading achieven^ent, by grade level, for Hmited^En^ish* 
speaking students in school programs supported by consolidated funding sources, 1976-77 
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Fijg, V-16. Prete!st and post-test standard ^^ores in mathematics achievement, by grade fevel, for limited- 
English^peaking students in school programs supported by. con^lidated fun^ding sources, 1976-77 
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The limited«Englishtspeaking students' post-test 
scores were also below the national average at aH-> 
grade levels. These and the gain scores between the 
pretest and post-test are perhaps even more diffi- 
cult to interpret than the pretext scores. Without 
more information it is impossible lo, determine 
^-^^adiether 'a student's improvement between the 
pretest and the post-test was the result of improve-* 
ment in the skill areas that the test was designed to' 
• cover ovjn English-language fluency skills. Most 
likely, the gain was the result. of improvement in 
both language fluency and achievement. Whatever 
these gains represent, ;the. pattern they show is' 
"encouraging^ The ra^e of growth between the 
pretest and the post-test was getter than the usual 
growth shown in one year at af^adfe levels except 
grades^ne and ten. 

Mathematics. Figure V-16 shows that the mathe- 
matics scores of limited-^nglish^peaking students 
.were below t^e national average at all grade levels 
on both the pretest and the post-test. The r^der 
should note, however, that at most grade levels 
these mathematics scores were several points higher 
than the corresponding reading te^t scores. In part, 
this* difference carl be attributed to the fact that 
.students with limited language fluency can answer 
more of the items on the mathematics test because 
Ifess reading is required than on the reading test. At 
all grade levels the gain score's >Vere above the 
• national norms, and at most grade levels they were - 
substantially higher. ^ 



Schools Tiiat Received Only Bilingual feducation Funds 

Most schools that received bilingual education 
funds also received support from one or more of 
the oth^r consolidated^-funding. sources. A few 
schools, however, received only bilingual education 
funds. Figures V-17 alid V-18 show the readmg'and 
mathematics achievement, respectively, oMmiited- 
English-speak^ing students who participated in 
school {programs supported by bilingual education' . . 
funds only. So^ few students participate/1 in these 
school programs-at most grade levels fewer than 
ten schools are represented -that no attempt 
should bd. made to draw any statewide program 
inferences. In both reading and mathematics, 
pretest and' post-test scores were below^- the - 
national average, and the g^in scores, while in most 
cases positive, showed no consistent pattern. 

Schools That Received Only Miller-Unnih Funds 

Schools that receive funds only through the 
Miller-Uj;iruh Basic Reading Act are not required to 
submit pretest and posWest achievement ^ata for * 
participating, students. Thus, jio separate infor- 
mation 'on the^bperformance of such, students is 
available for analysis. Schools that received Miller- 
Unruh funds and funds' throu^ other consolidated 
application funding sources -submitted pretest anci 
post-test student achievement data for all partici- ^ 
pating students. The analysis of* ihese data is 
presented in the sections entitled ."All Schopls with 
Consolidated ApplicationJFunding'' and ''Multiple- 
Funded Schools." 

< \ 
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Fig, V n. Pretest and post-test stjmdard scores in rt^ding achievemertt, by grade level, for Kmited-Enghsh- 
, speaking students fai school progranfe supported by bilingual education funds*only , 1976-77 
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Ffe. V-18. Pretest and post-test standard scores m mathematics achievement, by gradeMcvel, for lunited- 
* English-speaking students in school programs supported by bilingual education funds only, 1976-77 . 
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VI. Special Findings 



Evaluation Report of 
y. 1975-76 indicated 



that in some schools, third grade feadihg scofes, as 
determined b> the California Assessment Program 
were. declining. Specifically, the findings indicated 
that "in schools whose entering students averaged 
between the 21st and the 99th percentiles on the* 
1973-74 Entry Level Tesr, ^grqde* three reading 
achievement improved mark'Cdl> beyond predicted 
levels after three years in ECE^V In contrast, "in 
schools whose entering students averaged below 
the 20th percentile on the 1973-74 Entry Level 
Test, grade three reading achievement' declined 
relative to prediction after three years ijl<^CE/'^ 

p Among the schook whose entering^lidents 
averaged ll?low the 20th percentile on thg 1973-74 
^E/itry Level Test, not all had grade three reading 
scores that each year were injcreSsingly lower than 
the predicted average score. In some schools grade 
three reading achievement scores increased relative 
to the predicted score over the three-year p^od. 
In fact, of the schools whose students averaged in 
the lowest 2p perpent on the* 1973-74'tnrr)' Level 
Test, 1 10 showed increases in residual scores, 146 
■showed decreases, and 121 showed no change. 

While the overridi^ purpose of the Depart- 
ment's special study and the research conducted 
SRT .International "was to determine why the 
average scores declined for some schools below the 
20th percentile on the Entry level Test, informa- 
tion about schools whose average scores Increased 
was als|) considered. 

• Perhaps the most important question dealt with 
in the si?ecial study was "WTiat circumstances are 
associated with decline or improvement in third 
grade student reading Achievement?'' Two comple- 
mentary research strategies were selected for use in 
answering this' • question* (1) traditional 



^Evahtation Report qf ECE, ESEj^JUle I, and EDY, 1975-76. 
^nm«nto 'Califofnia State Department of Education, 1977, p. 
36. 

% Evcmatioix Ijieport, p, 36. 



hypothesis-testing study of variables witjiiri%ass- 
rooms, which was conducted by SRI' International, 
and (2) a series, of case studies of 16 schools-eigKt 
schools with increasmg average third grade readmg 
scores and eight schools with decreasmg scores- 
conducted by the -Department, 

The SRI ' International study was designed to 
identify classroom processed related to changing 
test scores among the schools whose students 
averaged in the Ig^est 20 percent on the 1973-74 
Entry Level Test. The research strategy employed 
in the study provided for identification of relation- 
ships between processes \vithin classrooms and 
student achievement, but it did not provide for an 
J examifiation of processes at the school level. 
Tfierefore, a second research strategy, the school 
case studies conducted by the Department, was 
employed. , ^ ^ 

So that the fin4ings^wpuld. " represent the 
strengths of both strategies, §ui gverlappmg sample 
of schools was used in the stupes. Seven of the 14 
schools jn the SRI International study were also 
included in the Department of Education's- case 
studies., AH school^ in both studies were also ECE 
schoc^ , . . 

^The informatirti'that follow? represents a sum- 
mary of tl^ findings from both studies, A detailed 
report of *e studies will be published separately a^ 
a companion document. to this report. 

, Special Case Studies 

Department of Xdu<iation personnel, working in 
pairs as obserVei^, conducted spetial case studies at 
16 sqhobls! The schools were selected on the basis 
of sexeral criteVia, including (1) having been m the, 
lowest 20 percent of schools on the 1973-74 £'n/r> 
Level Test, and (2) having shown an increase or a 
decline in thifd grade reading achievement on tjie 
California Assessment 'Program tests from 1973 to 
1976. ^ ' . ' 

An individual case study report was prepared f(Sr 
each schools^ visited. Each report included the 
observers', explanation of why test scores had 
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changed in the school. Findings and implications 
across schools were developed later. 

Several limitaUons of the case study method- , 
ology impinge *upon the interpretation of the 
'^outcomes of the ca^ studies. Qeneralizations 
der^ed from the 'Studies ^e limited to the popula- 
tion of the schools in the study sample, however, 
both the findings and the 'i^nplication'S may be 
apphca'ble to a broader population. 

Another limitation is the reconstruction of past 
events. The usefulness of the -reconstruction was 
dependent upon the .quality and quantity of 
historical data gathered in the schools, which, in 
turn, were subject lo, varying degrees of staff 
t,ui7ipver and accuracy cjf memory. 

An important' distinction should also be noted 
between a case study and a traditional statistical 
study.- In the case stuc^y, unlike the statistical 
study, the significance of the findings and implica- ' 
tions'is not relatetfto the frequency with which an 
event occurred. 

The major findings of the case studies and their 
implications are presented below under the follow- 
ing headings (1) Understan(jmg of the Intent of — 
Early Childhood Education; (2) Leadership and 
Management of Change; (3) Expectations of 
Teachers and Administrators-: (4) Staff Develop- 
meJit: (5) Teaching and Learning in the Instruc- 
tional Program; (6) Curriculum: and (7) Evalua- 
tion. These topics were selected because of their 
^uniqueness. within schools and their commonality 
across schools and because t^e observers believed 
that aspects of each affected student achievement 
in some way. ' . « 

Understanding of the Intent of Early Childhood Education 

,The findings related to understanding of the 
intent of early childhood educatio*n and the impli- 
cations of those findings are *scussed below. 

Findings The observers determined that the 
translation of the intent of early childhood educa- 
tion into educational experiences for children was . 
impeded by the lack of understanding of many 
people about the intent* of ECE. Breakdowns in 
communication were found to be common in 
schools with* decreasing' average reading achieve- ' 
ment scores.^ In schools with increasing average 
scores, communicati^on breakdowns were far less 
frequent. In* most* schools with increasing average 
score?, the observers note'd a^ general sense of 
educational purpose that supported soine of the 
specific purposes of ECE, Some other findings 
were the follgwing: • • . 



1.. TeaclTers, principals, parents, and children did 
things only because they beheved that some- 
one else' (the state,- for example) required 
: them to do so, they did not see themselves 
functioning in terms of an intended purpose. 
For example, some individuals s^w the school- 
level plan as a proposal required to obtam 
, outside funding rather than as the documenta- 
tion of a planning process, the purpose of 
v^ich was to improve ^he effectiveness of the 
educational program. 

2. Teachers,, principals, and parents wanted -to 
comply with such exhortation^ as "meet the 
needs of each chjld,'' but they^did not know 
how to do ^. 

3. Incentives that were designed at the state level 
to effect reinforcement of the intent of ECE 
were misused for unrelated local purposes 
thaf'-^ften were tied to school or distnct 
politics or to the priorities of persons in key* 
positions. 

4. Some information 'was not available to people 
with operational responsibihty, such as the 

" pnncipal, teachers, ' and aides. Communica- 
tions from ^tate,' district, and staff develop- 
ment « program personnel tended to be 
. abstract, impersonal, and difficult to relate to 
problems in the classroom and the school. ' 

Implications. The implications of the above 
findings are as follows: ^ 

• School, distric^, and state accountability pro- 
cedures- shoulti focus more sharply on the 

^"^ purposes of educationjn general and of ECE 
in particular.' Too often, accountability be- 
comes a matter of' doing so?netl*ng to "look 
good^" .^uch ^s keeping records on students 
b.ut not using them. , »^ ^ 

• Supportive services to teachers, aides, princi- 
pals,, an* other school^ staff should^ be 
designed more carefully for their respective 
users. Many school personnel need to be told 
"how to" and in terms with whigh they are 
familiar. They need to know ^hat to do in 
their own worlds. Teachers, parents, and 
principal? need to be better informed about 

9 ECE-what the options -are, what the responsi- 
bilities are, what services are available, and 
generally wha^to expect. , » . 

• Communication-whether wntten or oral, 
frdm the state, from tl\e distfict, or from 
witPKn the school-should be mpre frank and' 
.personal, and the emphasis shouldN^e on the 

.j^ primary role of human beings in the educa- 
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tional ptocesses. ^too much of th,e infor- 
matibn received 'by teachefs^ parents, and 
Children is couched in the abstract^anguage of 
"programs/' which does nop help people 
understand what they are to do\ ' ^ 

Leadership and Management of Change 

The findings related to leadership and manage- 
ment of change ^and the implications of those 
findings arp discussed below. 

Findings. The importance of leadership in 
managing change effectively vvas demonstrated at 
both the school and district levels. In three school 
with increasing student achievement in reading, 
district-l^vel leadership served as a supportive force 
in the introduction and implementation ®f new 
progtams, at a school with decreasing achie^ment, 
district-level leadership proved to be an obstacle or 
disruptive force. School-level leadership ejnerged 
frequently as either a positive or negative factor. ^ 

In four schools with i^acreasing achievement, the 
^principal clearly ^hibited positive leadership; in 
two schools groups of teachers assumed school- 



other things, close contact and irtvolvement 
with those who have operational responsi- 
bility for implementing decisions, such as 
teachers and aides; mutual trust and respect;* 
anticipatory planning; and support for on- 
going skills development. These and other 
good leadership characteristics should be 
fostered through personnel practid^s and 
policies, especially through the appointment 
of school principals, and, through staff devel- 
opment programs specifically designed to 
improve the leadership capabilities of those in 
leadership roles at the school, district, and 
state levels. 

• The' leadership potential of a core group of 
teachers at a school should be acknowledged 
and supported where possible, perhaps 

^ through the participatory planning features of 
^ ECE. 

• The role of tlie district in improving school 
programs should be clarified. 

Expectations of Teachers and Administrators 



level leadership in mana^ng change, and in one 
^school the resource teacher emerged as the leader. 
^Characteristics pf positive^ leadership appeared to 
beVl) w^espread involvement and delegation of 
decisKm-making resppnsibility* to those charged 
with' iihplementing decisions; (2) mutual trust in 
and respasl for the competence and judgment of 
• others; (3)\nticipatory planning; and (4) support 
. for oi]jfDing skills development and learning. . 

By contrast, at "^ost of the , schools in which 
achieve me a|^ad dechned, either no leadership or 
negative leadership characteristic? were noted. 
Authoritarian principals created an atmosphere of 
repression at iwo schools. In other ^^SKocffs the 
absei:ice of effective leadership was illustrated by a 
lack of planning for changing school populations 
(particularly for larger influxes of Spanisji-speaking 
pupils)' until a crisis situation developed and by a 
lack of leadership in matters pertaining to the^ 
selection and implementation of curricula ^d 
classroom organization. 

, Implications, The inipKcations .of the above 
findings are as follows: ' , \ 

• Changing school programs mea^, above ail, 
changing people's ways of doin| things. Good 
leadership, regardless of its souV^e (principal, 
reading coordinator, teachers, or district-level 
personnel) Is fundamental. The' potential 
damage from poor leadership is equally 
important. Good leadership entails, among 



ihe findings related to teachers' expectations 
and the impUcations of those findings are discussed 
below. 

Findings. From their schopl visits the observers 
concluded that teachers' expectations for and 
behefs about the children they taught often seemed \ 
to influence the children's learning and that 
administrators' expectations of teache/s' perfor- 
mance seemed to affect the performance j^of 
teachers. 

The expectations of tfeachers for their pupils 
were reflected in vano\is ways. (1) their percep- 
tions of the childreirand their abilities'; (2) their 
selection of curricular material, and (3) their 

^ standards for pupil performance. In schools in 
which teachers believed that the children had poor 
otal language skills hnd "many problems'," the 
children performed poorly. At a school in which a 
teacher referred to non-English-speakjng, children 
as bflfimg bright and learning quickly, the children 
performed well. Decreasing scores were noted in 
schools in which the curriculum required httle 
more than low-level tasks, such as rote responses. 
Increasing scores were noted ia Schools that had 
comprehensive curricula and in which mastery and 
competence w^re expected and demanded. 

Where teachers were held accountable for the^r 
pupilsl performance , and given accurate rnforma- 
about that performance, their behavior was 

^nore likely to have a positive effect on pupils' test 
scores. \ 
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The differences in the characteristics and back- 
grounds of the children at the schools visited were 
great, but the differences in teachers' perceptions 
of and beliefs about .the children were even greater. 
The observers' con^e^ded that the attitudes and 
expectations of teachfe^s and other adults were 
Significant^factors in pypils' improving or declining 
in reading performartte. 

Implications, The implications ^f the above 
findings are as follows: • - / 

• The expectations for what children can learn 
need to be raised in many schools, especially 
those with a high minority population. This 

. includes the teachers' expectations with regard 
to the following: 

1 . The types of activities (reading and writing 
words, sente^ices, and paragraphs rather 
than just filling in the blanks, making rote 
responses, playing games,^doing puzzles, 
and the hke) . . 

2. The pace of each child's progress (each 
child njftfe^g optimal progress consistent 



-- - - 

with thr child's development rather than 
each child working as slowly as he or she 
desires) • 
3. The quality of the children's work (regard- 
less of the type or level of work, good 
workmanship rather than half-hearted, 
sloppy efforts) 

Staff bevelopment ^ , ^ « 

The findings related to staff development and" 
the implications of those findings are described 
'below. 

Findings. Three basic types of staff development 
programs Were notedm^e case'^ studies: 

1 . Ongoing inseTVice training closely tied to the 
instructional program. In five schools this 
type of program was found to have an effect 
on staff behavior and classroom practft;^es. The 
programs had a strong "how to" emphasis and 
were an integral part of t\\e operation, of the 
schgpl. Extensive classroorfi follow-up was 
also conducted. In three schools the qbservers 
noted that the inservice trainingtorogr^m also 
had a positive ^ffept on pupil pferQ^rmance; in 
two schools thdy determined tMt the impact , 
on pupil performance was neg^ive. 

The programs that had a positive inflybhce 
on pupil pefformance were characterized by 
the following: /(a) the training helped the 
teachers adapt ^ their existihg, instructional 



practices to the new pro-am; (b) the training 
program was adequately comprehensive; and 
• (c) the teachers' were committed to the 
program. By contrast, the two ineffective' 
staff development programs were "forced on" 
I'eluctant and antagonistic teaching staffs by 
authoritarian administrators. 

The effectiveness of ongoing inservice train- 
inj^ programs that were closely tied to the 
instructional program was attributable, to a 
large degree, to the fact that they were 
people-specific; that is, aides were trained for * 
the aide's job, and' teachers were trained for^ 
the teacher's job. The training included details 
ab6ut what to do in th^^assroom. Training 
was< provided in frequent sessions throughout 
the school year. 

2. Inservice training designed to improve the 
staffs abilities to serve ethnic and language 
minorities. Most of the schools, that were 
studied had significant minority populations, 
but at , only four did the qbservers note 

^ inservice training programs d esigned to 
improve statf members' abilities to serve 
ethnic and language minorities. The observers 
did note that courses in multicultural under- 
standing had a positive effect on the achieve- 
^ ment of limited- and non-English-speaking 
students. Three schools offered Spanish lan- 
guage courses for teachers. ^ 

3. Inservice training* programs on a variety of 
topics that were not specificqlly related to 
teachers' responsibilities. These were non- 
specific presentations that were ■ usually 

' designed^ for a general audience of educators. 
\ J Theu" impact' on teacher behavior and perfor- 
' " * mance was negligible. 

Although iiDt} a ^rm of inservice training, " 
informal exch^gey^mong teachers as they worked 
together to plan the kindergarten through grade 
three program wer^ found to be of significant 
val^,. The world of the self-contained classroom 
^ has been opened up to fellqw teachers, aides, and. 
parents. In some instances this opejiness, fostered 
an increase in. the^fiaring of ideas, techniques, an3 
materials and i*n commitment. In other cases this 
ORennes* resulted in insecurity, retrenchment, and 
a sharing of excuses. 

Implications, The* imphcations of. the abqve 
findings are as follow^: 

# Staff development programs at the school 
level should be .more cloSely tied to the 
' *instrucMonal prdgram. Follow-up in^the form 



ERIC 
1 



69 




of observations in the classroom and feedback 
to teachers and aides on their performance 
should be included. The emphaslj^ should be 
on helping teachers and aides to do their jobs 
better. Ideally, training sessions should be 
conducted frequently and should inJude 
demonstrations involving children. 

Staff development should be an integral part 
of school operations, not merely an additional 
or external actmty. 

Inservice training should be designed to help 
teachers integrat^ new programs with existing 
practices. 

Staff deyelopmefit activities should not focus 
on any curricplar area to the exclusion of 
others. 

Stafg development programs should foster a. 
clear sense of purpose and conimitment to the 
program. 

Staff development' activities that are not 
specifically related to the planned instruc- 
tional program of the school should receive a 
IgAyer priority and i smaller 'allocation of 
resources than tho^^ activities that ,are 
directly related to the instructional program. 
Conferences, workshops on tangential topics, 
workshops of a general nature, and outside 
consultants not familiar with the school pro= 
gram diould supplement the basic staff devel- 
opnfeht effort. • * * * 

Personnel at the state, county, district, and 
school levels 'should work together to identify 
the kinds of know-how that teachers and 
principals need and to ensure that such^ 
knowledge is widespread. 
Workshops at all levels should be devoted 
equally to providing know-how and clarifying 
purpose in relat\on to the content area. 



Teaching and Learning in^he Instructional Program 

The finding related to teaching and learning In 
the instructional prografn and the implications of 
those findings are described below. 

Findings, Individualization of instructional pro- 
grams had positive effects on student achievement 
in those instances in^ which people provided 
instriictioa. Negative effects were noted in thqse 
cases in which instruction was provided primarily 
by means of materials, Such as self-teaching mate- 
rials. ■ 

In schools in which test scores increased, the 
observers noted the following: (\) initial tea6hingV 
was done by people rather than by m^ns 0( ; 



media, materials, or learning stations, (2) diagnosis 
of pupil needs was an integral part of the program, 
i)ot an external activity, (3) prescription flowed 
from diagnosis and included instruction by teachers 
(or aides) rather than just a' matching ofAupils 
and materials; and (4) monitoring of the chilH^n's 
ability to use new concepts was a part of the 
instructional program. / 

Student performance improved in those schools 
in which learning centers were used for purposes of 
reinforfcng students' skills. No success was noted in 
tho^e sfchools u\ which teachers relied on curricular 
materials far instruction (as in a correspondence 
course). When children's work was not corrected or 
evaluated for long periods of time, their early 
misunderstandings were-rein forced through repeti- 
tion, which resulted in poor performance. 

Implicatk>ns The implications of the above 
findings are as fpllows: 

• Initial teaching should Ipe done by the teacher 
(or some other person) rather than by riTeans 
of media, materials, or leamiilg statidhs. 
Diagnosis should be an integral part of te; 

' jng . and learning rather- than ' an external 
* activity. The teacher should pay close atten-^ 
tion to how each child responds to instruction^ 
and should make occasional infom}^l assess- 
"ments of the child's progress. The teacher . 
should use the children's work as a printSfy 
source pf diagnostic information. , 

• Prescriptions should generally include instruc;- 
tion by the teacher; they should not be* 
limited to a designation of appropriate 

• instructional materials. 

• The child's ability 'tg , use new skills and 
^ concepts in a variety of contexts, such as 

, using readihg skills to reacf a ^science book, 
should be fnonitored. Reteaching should be 
provided^whep appropriate. 

Curriculum 

The findings related to curriculjim and the 
implioations of those findings are described below. 

- ^Findings. Reading scores declined • in those 
schools in which the curriculum consisted entirely . 
or mostly of reading. Scores increased in those 
' schools in which students had ample opportunity^*^ 
to apply their reading skills in other curriqplar 
argbs. The schools in which reading scores increased 
were characterized by the. following. (1) ^teachers 
^ were Committed' to the programs, (2) the content 
' of the curriculum was appropriate to th^ particular 
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needs of the pupils; for example, the content was 
sufficiently comprehensive and could be adapted 
to meet the spe,ci^l needs of pupils, including 
limited- and non-English-speaking children; and (3) 
the implementation of the curriculum was accom- , 
panied by effective staff development and follow- 
up activities, 

'A decline in reading scores was noted in tho^ 
schools ill which strong emphasis was placed on 
decoding and phonetic analysis and little emphasis 
was placed on such integrated skills as comprehen- 
sion afid composition. 

Implications, The implications of the above 
* findings are as follows: 

• Students should have ample opportunity to 
apply their reading skills* in a variety of 
curricular areas. 

• Within the reading program a variety of 
reading skills slpuld be taught. 

• Steps should be taken to ensure teacher 
_commitm_ent_ to programs. Jeachers should be 



programs. 

• The curriculum should be such that the needs 
oiall students can be me*. 

Evaluation ^ 

The findings related to evaluation and the 
implications of those findings are des{:ribed below: 

Findings, Four problem^^were noted in the area 
of evaluation in all schools m the study: (4) school 
perspnnel had minimal information or incorrect 
information about the tests and test results; (2) 
-iBchool personnel failed to use the infornlation that 
was available; (3) monitor ajid review ratings and 
test scores are assessments of different aspects of ^ 
the school operation^ and 'school personnel'had 
difficulty integrating and interpfeting both types 
of information; and (4) y^ineans "available for 
identifying aind a^seksjng the progress of limited- 
English-speal^mg and\ion-English-sp^king chilcken 
arid for evaluating bilingual education progfanis' 
were inadequate. * ^ 

Personnel at several schools could not recall the 
California Assessment Program reading tests, and 
none' of the teachers who were interviewed had 
seen the test results. Many teachers and administra- 
tdrs did not distinguish between the Qlifornia 
Assessment Program tests, and the standardized 
tests that they administered. In two instances 
school personnel believed test scores had decreased 
when, ^in fact, they had increa'sed. In another 
school the reverse of this situation was noted. ^ 



Observers found only one instance iri which 
program had been modified as a result of either the 
monitor and review evaluation or the California 
Assessment Program test results. 

The use of an English-language test tO' measure 
the progress of students ^hose primary jeading 
instruction was provided in Spanish was found to 
serious problem. 

implications. The implications of the above 
findings are as follows: ^ 

\^ School personnel should be instructed iri how 
to use tesis and test results in the district, in 
the school/ and in the classroom. 

• The* relatfonship of the strategic implementa- 
tion components of ECE-^planning, imple- 
mentation, and internal and external program 
quality review^ne€ds to be stated more 
explicitly. . > \ , 

• The relationship between program manage- 
ment and ©valuation should be clarified at the 

"school level. ScTiool personnel do not seem 
V ^ view pro grams "in terms of people arid 
^ behavior;s. ^ 
, In some schools with full bilingual education 
programs, heavy 'emphasis is placed on 
instruction in the students' primary langua'ge 
\ during their early years of elementary school; 
in 'the later years emphasis is placeri^ on 
instruction in JEngiish. A need exists for some 
way of a^essing the students' progrejss during 
■ both phaseS'Of thejr e'ducation.. 

Classroom Processes Evaluation Study 

The study conducted by SRI Intejrnational was 
designed •'to, examine classrooqi prpcesses that 
might be relatedilb studgnts' reading achievement. 
The study sample,, wh^p consisted* of 45 thfrd- 
grSde 'classroom^ in 14 schools, was drawn fronl 
the same ^oup of schools that the Department 
used in its special case studfes. All schools had been 
participants ir> ECE since 1973 of 1974. The 
average score of each school on the 1973-74 Entry 
Level Test was at or below^the 20th percentile. 
Half the schools had had increa^ng residual scores^ 
on the California Assessment Program third grade 
reading test over a three-year period, and half had 

. had decreasing residual scores over the same 
period. Theischools were^'also selected qn the basis 

, of high or low concentrations of.^ limited- and 
non7English-speaking chilldren. 

Car/sfully trained obsepers. spent two days m 
each third grade classroom. They recorded data on 
the Classroom Observation In^itrument. This ihstru- 
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ment allowed for the collection of three types of 
information/ 

\. Classroom summary information-Chssvoom 
sumniary information included the number of 
^ children enrolled, the number present on the 
observation day, the number of teachers and 
aides assigned to the classroom, tjie number 
of volunteers or visitors present, and the 
length of the school day. ' 
2. Physical envirdnment m/orma//ow-Physical 
environment information pertained to the 
cfassroom setting, including the presence and 
use of specific 'equipment, instructional mate-: 
rials, games, toys, and displays; the 'types of. 



desks and tables and their patterns oFjpIace^ 
ment; pupil seating (whether assigned or 
self-selected); and pupil groups (whether 
assigned or self-selected). 
3. Classroom procedure information— Chssroom 
procedure information dealt with classroom 
processes and structure and interaction in the 
^classroom. Activities were described in terms 

-of-the-^str ibution of c hit dren anU ad ults at a 

particular time, child and adult grouping 
patterns, adult roles, child involve^ment, and 
materials used. A five-minute observation was 
made of the activities, grouping, and inter- 
' action of a preselected "focus" person; and a 
five-minute process observation was made of 
that pei^n.. The observations were made 
several times each hour. 

'The informatidai derived from the observatiohs 
* included^ information about who did what to 
whom and how. From the observations a large 
number* of classroom process variables were iden- 
tified. From California Assessment Progrannyi^read- 
ing test ^ata; a reading achievement dependent- 
measure w^s calculated for eadi olassroom in ih^ 
study. Also, an average ^ absence rate was deter- 
mined for each classroom from j)upil attendance 
data. The achievement measure and absen^ rates 
were each analyzed in' terms of "their relationships 
wittl the classroom process var^bles. 

" \ ■ ■ . . ' • 

Findings , * ' ^ ^ 

Several analyses were carried out fo study the 
instructional processes used in the sample class- 
rooms. Schools with increasing average scores were 
compared with those with decreasing seore;Sv Class- 
room process variables, and reading score correla- 
tional patterns were examined. Because classrooms 
within the sample varied greatly with respect to 
reading pretest scores and with respect to gain 
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scores, subgroups within the sa'mple were cou]- 
. pared. Finally^ the relationship between absence 
rates and instructional process variables w^ 
examip^d. 

The 20 classrooms in schools with increasing 
scores were compared to the 25 classrooms in 
schools with decreasing scores to determine the 
^^tive occurrence of the process variables in the 
two sets of classrooms. Although 76 variables were 
found to differ significantly between the two 
groups, two problems made interpreting, the differ- 
ences difficult."* First, although efforts were made 
td determine matched samples of schools, demo- 
graphic differences between the samples did exist, 
and statistical 'adjustments ,forihe effect^-ofHhe- 
differences were impossible to make. Second, and 
most important were the significant differences 
within schools (bet\yeen classrooms) in both 
achkvemenf and process variables. When the differ- 
ejwJes between qlassrooms within schools are 

^eater than those between groups of schools, 
fhose between ' groups of schools cannot be 
explained as readily. ; 

Several analyses of the relationships between 
reading achievement^ scores and classroom instruc- 
tional process vari^fes were conducted. Since wade 
variations existed among tfie classrooms studied in 
terms of pretest (in this case, second grade) reading 
scores, the pret^t differences were taken into 
account in the analyses. • ' 

Within the classrooms studied the lower the 
ratio of students to adults was^, the higher, the 
achievement test scores were. How teachers 
organized their classrooips, used aide^, and grouped 
children had a relationship to reading achievement. 
The data indicated that teachers should assign 

, students to work groups rather than alloW them to 
select their own groups or place5^to sit during work 
time. Scores were higher in •classrooms in" which 
adults worked with children* in small groups than^ 
they were in those in which adults worked with 
children on a one-to-one basis. This finding does 
not mean that teachers 6riaides should not work 
with one child at a tinie; it indicates only that 
within a normally functioning classroom, effi- 
ciency* is not achieved' when much bf the riding 
time involves working with only one studenT at a 
time. In many instances, when adults spend a great 
de^l of time Nvith'' individuals, other children are 

i,not able to continue their reading tasks because 
thej^ lack the necessary direction and supervision. 
For some of the time, teachers w^ere not i^nvolved, 
with the children. They graded papers, prepared 
lessons, talked wifh other adults, prepared' art 



materials, or were out of the room. In those 
classrooms in which spcH situations occurred most 
oUten; scores were low' ^ 

Activities and materials were also analyzed in 
terms of reading achievement. The more time 
chUdren spent sharing ideas anc^ participating in 
g^oup discussions, the lower their post-test scores 
were. One should not infer that time should not be' 
spent in group discussions. Th? time spent in such 
activities should riot be so great as to leave too 
little time for reading. The more time children read 
or the more time that was spent oh reading, ^ the ♦ 
higher the test scores were. J 
Several materials or resources had- a positive 



rclaiiunship wiin me pQst-te ^ scores, including 
instructional charts, achievement charts, and 
instructional games. Basic equipment and instruc- 
tional materials were significantly related to gains. 
The use of noninstructional games was negatively^ 
related to achievement gains. 

A third area in which classroom vafiables were 
analyzed in terms of reading achievement was the 
types of interactions that occurred between teacher 
and child. 

A very direct method of instruction had a 
positive correlation with the po^t-test score. In 
such a method the teacher provides insti^uction and 
asks the children direct questions about the sub- 
ject, the chUdren respond, land the teacher provides 
feedback, letting each child know whether the 
response was acceptable. The correlation between 
reading 'achievement and acknowledgment and 
corrective feedback for responses to qqestions was 
especially high. ^ 

The more chDdren asked questions, the higher 
their scores were. Scores were higher in those 
classrooms in which adults responded with a» direct 
question than they were in those classes in which 
adults responded by supplying the answer. , 

In those instances in which adults and children 
were most often invoJySd in nonacademic inter- ^ 
actions, the readina^est scqx^s were low. Test 
scores were also /low. in . 
chUdren eihibiteS negative oj 

For another analysis clas^ 
on the basis^ of the level of tl 
scores (students" average', score) and the average 
gain in score between the pretest and the post-test. 
Two groups, bath of which had medium to high 
pretest scores, were selected for comparison. One 
group showed little or no gain fport the pretest to 
the post-test. 

The other group did show some gain. The group 
that showed gains differed from the group that 
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showed no gains in a number of ways. The average 
class size in the no-gain group was twice that of the 
gain group. The ratio' of students to teacher was 
much lower in the gain group (approximately 6:1) 
than it was in the no-gain group (approximately 
13.1). The difference in this ratio could account 
for much of the difference in test scores, but the 
manner in which the teachers and aides worked ' 
with the children was of primary importance. 

The two groups^ffered in terms of instruc- 
jtional processes in sevi^al important ways. In the 
gain group the teachers generally provided instruc- 
tion to' small groups and used a direct approach in 
their teachihg. They made great use of educational 
equipment and materials,* including .audiovisual 
equipment. During nonac^emic activities they ' 
often asked open-end^d questions, and the children 
' often made extended responses. 

In the no-g^in classrooms, the teachers often 
worked with one^thilcl at a time, and the other 
children \yorked independently. Arts and crafts 
and drama activities were more frequent. Teachers 
w§re less often directly involved with children, and 
the children exhibited more negative behavior. 

A high rate of absence was a problem for 
teachers and students. The average absence Vates 
for the 45 classrooms ranged from 1.0 to 22.5 day{ 
per student. The relationships between classroom 
variables and student absence^ates were analyzed. 

High pretest and post-test scores were noted in.' 
classroom that had low absence rates. 

The number of aides in the classrgiom and the ' 
, ratio of students^ to adults was related to the 
student, absence Vate- -The lower the ratio of 
students to adults, the lo\vev il^ absence Tate.. 
More absences were noted inNJ^^^oms in which 
stationary dpsks were placfed in roWs and students 
'were assigned to seats .than ^F^re noted in classes 
with^different seating patterns and feqijirements. 

Several variables in the organization of student • 
groups and^tlje responsibilities of. the adults were 
relied to absence Vates. ^Children in classrooms in 
which the adults worked with one child at a time 
were absent less often t^an those in classrooms in . 
which the. adults wpriced with children on most 
oiher bases. StudeYits were also absent less often in 
classrooms in which adults 6bserved often or 
attended to small groups of children. An appar- 
ently contradictory finding was that in classrooms 
in which childreh oftea worked* independently in 
either reading or math, the absence rate was low. 
From this finding one can infer that children 
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enjoyed, personalized attentWn but also enjoyed 
working independently. 

> *^ 

Activities , and periods that were ^ less> task- 
oriented and more social in nature-^oup time, 
transition, snack time, and lunchthpe-had a posi- 
tive relationship Avith absence ratei^The greater the 
occurrence of such activities ^d periods, the 
higher the absence rate. Also, absence rat^s were 
bd?st in those classes in which adults weH, 
frequently involved with* children and ^hildren 
were frequently involved in activities. Reading and 
math activities had a negative relationship, with 
absence. Children in classrooms in which rigorous 
academic activities were conducted were ahsent 
less often ih&f^ children in less academically 
oriented classrooms. 

'% 

Attendance appeared to be better in classrooms 
that included a business-like,, structured approach 
to education than in less structured classrooms. 
Verbal interactions of an academic nature had a 
positive , relationship with school attendance. 
Interestingly , a more relaxed interaction pattern in 
which childr^ initiated and adults responded was 
negatively related to the absence rate. The more 
adults had to control behavior and the uhhappier 
children said they were, the higher the absence rate 
was. 

The variables present in a mbre^ structured, 
teacher-directed approach to education had a 
'posit^ correlation with reading post-test scores. 
Thus, pretest and post-test scores were highest in 
those classrooms -in which the absence rate was 
lowest. 

Other ECE variables, such as the presence of 
aides in the classrooms, had a positive association 
with reading post-test scores and a low absence 
rate. IndividuaUzed attention to children was asso- 
ciated with a low absence rate but not with high or 
low reading test scores. A Variety of instructional 



materials was associate 3 with reading achievement 
ajad a low absence rate. 

Conclusions 

The in-depth study of selected ECE classrooms 
indicated that the instructional methods or pro- 
cesses used in some classrooms were more efficient 
in teaching reading than ^thosej used in .other 
classrooms. Of particular importance were the 
findings that gains were greatest and the absence 
\rate lowest where the student to adults ratio was^ 
lowest. r 

However, merely having a number ^'of adults in 
the classroom did not improve student progress. 
The manner in which groupsrwere organized and 
managed was of critical importance. Teachers in 
the classrooms in which gains wdre small often 
instructed one child at a time even though the ratio 
of adults to children wasqyite high. This organiza- 
tion resulted iji the other students' working for . 
long periods of time on their own. Also* children in^ 
Classrooms organized in this manner exhibited 
negative behavior more 'often than those in class- 
rooms organized in other manners. One ^ staff 
person working with orje student at a time during 
the ^class pading period did not appear to be an 
efficient strategy. , 

The ^eatest gains in reading wer« made hy 
students in those classrooms in whic5h adults often 
worked with small groups o/ students and often- 
used direct teaching ipethods (providing instruc- 
tion and asking questions or asking students to 
read aloud). In such classrooms the students 
received immediate feedback for their responses. 
a response was correct or satisfactory, the student 
received acknowledgment or praise', and if it was 
tncorrect, the student was guided to a correct 
response. Teachers i!j these classrooms were very 
successful in kegging studepts working on the 
desired tasks. Conse^HentJyfT?ty Uttle iftisbehavior 
was noted in these classrooms. 
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VIL Locally Developed Evaluation Reports 
' vof\gecial Programs 



, Fof several years the Department of Education's 
annua? evaluation report of special programs has 
included finiiings'fcased~on (l);d^ta submitted by 
^tricts on 'liie Consolidated Evaluation Report 
forms (Form E-127P); and (2) the results of the 
California Assessment Program tests. These data 

/have been augmented by findings from various 
special studies undertaken by the Department in an 
attempt to add depth to the relatively narrow view 
of special* programs obtainable frtoi the limited 
common data that-^ere-available^romi* participat- 
ing schools. 

fiach. year school districts conduct evaluation 
studies to^ serve local needs. The findings from 
many of these studies ^apparently have value 
beyond serving local needs. To investigate hQw this 
resource of evaluation 4ata might be tapped, the 
Department ^ obtained and reviewed during late 
summer of 1977 copies of-- locally developed 
reports from a sample of'¥l districts. The districts 
varied in size from some of the smallest to somfe of 
the largest in the state. The Department received a 
total of 77 reports. A few were only two or three 
pagesf in length ;^some exceeded '40 pages. No 
district submitted an individual report for all of its 
schools. The large districts typically prepared the 
greatest number and kihds of special reports. 

The pattern of having both state-required reports 
on state-devised forms (Form E-127P) and locally 
designed and used reports of the evaluation of 
special programs has been developed over the'years 
in response to certain changes in the requirements 
for specially' funded programs. The evaluation of 
special projectS|Oyer the past 1 IVi years (beginning 
with the first ^luation of ESEA Titk J programs)" 
began wi^ a generalized approach. The programs 
in most districts were designed to be common to 
all schools withjn the distript. Consequently, the 
Department required only one locally developed 
evaluatioij. ^'report (written largely in Narrative 
form) for eacli commonly funded district program. 
The same report serve d bo th state and local' 
purposes, usually, limited evaluation data were 



also provided by the district' evaluator to each 
participating school, and a summary dsport was 
r^sented^o the district governing board. 
As it became apparent that'the locally developed 
reports were too varied and too voluminous to be 
practical for summarization at the state level, 
forms wer? devised for district use^ and common 
data were collected for state' use. 

In 1970 the evaluation of individual school 
programs became ^necessary because of new state 

requirements^^ « — 

. ' To alleviate the problems of districts in prepar- 
ing and ^bmitting multiple reports and to keepi 
the number of pages of reports at a manageable' 
, level for the* Department, the Department has 
markedly reduced the length of the required 
reports over the years. Much of the report now 
comes to the Department in a coded foj^ so that 
data processing services can be used. The data 
requested include an "accounting of participants, 
their standardized test scores, and a summary of 
the extent to which planned objectives were met. 
No narrative now accompanies the reports. 

The . Department has attempted, through a 
variety of ways, to promote thorough evaluation of 
district and schooj^jprograms at the Ipcal level. It 
has recognized that such evaluation, and reporting, 
must go bfeyond that which is required for comple- 
tion of the state forms. The kinds of data 
approi)riate for aggregation stateVide are inade- 
qua\e for purposes of program improvement *and 
needs assegsiy^t at the local level. Evaluation 
procedures and reporting formats must be adapted 
to fit the various^udiences (school staffs, parents, 
district office staffs, and governing boards) for 
whom they^ are intended. The Department has 
therefore encouraged districts to continue to pre- 
pare the narrative reports in addition to completing 
the state forms. . * ^ 

Types of Local Evaluation Reports 

Many .factors affect the content and type of 
locally^eveloped evaluation reports. First, varia- 
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tions are essential in reports or presentations to 
different audiences. Second, reports for similar 
audiences ii^y differ from di^rict to district 
because of ufhique circumstances. 
Typically, local evaluation reports fall into the 
I following categories: . 

>y • District reports to the governing board. Nearly 
all districts in the sample prepared a written 
report fo^ their governing boards. The report 
was in layperson's 'language and included 
information about the kinds of specially 
funded programs in the district and the results 
of tHe jjrograms. The reports of the districts 
in the sample were from 20-50 pages in 
length. In general, sections were devoted to 
background information, evaluation proce- 
dures, results, and conclusions. 
Individual school reports. Most districts in the 
sample also prepared a report for each partici- 
pating school. These reports were 5 to 20 
pages in length and consisted of a brief 
pfogwv description, tnformation abottt^the 
extent to wliich the school's objectives were 
met, a description of the solution procedures 
that were implemented, identificatioji of 
those procedures that were effective, and an 
analy^ of standardized test score?t*ot various 
groupings of students for the current year 
and, where possible, over several years (longi- 
tudinal data). 

Tesf score analysis reports,. Several districts ih 
Hhe sample prepared separate detailed analj{ses 
oK^tandardized test results. These typically 
inclimed comparisbns of current scores jvith 
scores from previojis years, confgarj^ons of 
results from different grades, and comparisons 
on aVubject-matter basis. ► 

• Evaluation abstracts: Some -districts prepared 
a separate evaluation al^tract that cQnsisted 

^ of a program summary. These included infor- 
. mation on goals, program sc^pe ^ and costs, 
plain features, and evaluation results, Also 
included were recommendations^or pi^gram 
improvements and implications. / , 

• Needs assessment^ data: Some-districts in the 
sample supplied to each schopl a report 
cpntajining both di^jtrictwide and in^vidual 
sci^odl needs assessment data- that^ could be 
used by the school's program {fanning com- 

^ ,mittee. 

^'Special studies: Most districts also reported 
on special studies that they had conducted 
throughout the year. Such studies were con-, 
ducted only periodically in some of the smait 



fislricts and on a regular basis in sdme of the 
TargeN districts. """More detailed information 
about^ the special stutiies.may be found in 
Ar^ndix 4. " - ^ 

An^xfensive amount of information is includi 
in the local evaluation reports. ^ 

^ JExampIes of Comments About Programs 

The following excerpts from the reports of the 
12 districts in the Department's sample illustrate 
just some of the types of informatioh provided in 
local evaluation reports. (See Appendix 4 for 
additional exdSrpts and comments.) 

9 

Impact of Programs 

Workshops in which tethers and aides 'worked 
together have been extren:\ely productive because they 
made items that they needed. Research and new ideas 
were introduced, and the latest materfals were made 
available for their review. ^ , 

Post-test grade equivalent scores for the . . . school 
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~7ear"were higher at nearly all grade levels for readmg and 
mathematics than thp previous year. In part, this may 
have been due to a change in test forms in the overall 
testing pro'gram. G^ins in reading and math from pretest 

■ to post-test for the total group of . . . students were also 
higher than for the total group of project students the 
pre\qous year. 

The . . . procedures . . . have been carried out so very 
effectively duetto the combined efforts of oiir outstand-' 
ing facililatbr, dedicated teachers, aides, and volunteers. 
As a parent and,an*aide, I feel the ECE program is 
meeting tlwn^arted educational needs of/oyr children. 

I cangot ima^nefThat a better educational experience 
for K-3 children ir available aiywhere. I*am particularly ' ' 
impressed t>y the thoroughness with wjiilch each objec- 
tive i§ defined, carried gut, and constantly evaluatedjf^ ' 
However, the school 'goes fat beyond each state^^ 
objectLve-the program is even better than it appears ir> /\ 
writing. ^ \ ^ . ' . 

There was>an improv^^ment in this, yfi^r^atlqid^a^e* 
rate over last^ye^V * ^ 

The students became interested in irfath operations 
activities because they Jiad use\f calculators to- v^.^ 
check their work themselves without waiting 'for the 
teacher or instructional assistant.^There. was enough 
challenge for the students tcxydo the work correctly so 
that they could come up with the answer that was on 
their calculators. » This project was so successful that ^ 
plans call for expanded use [of calculators] in^all of the 
sixth grade classes next year. 

Given the standardized test data available, the Title 
I/SB 90 program clearly can be ^d to haye been 
effective in meeting academic objectives.*. . . At each of 
the K-3 grade levels, the 17 pefcent'net shift objective 
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was met, and median percentile score gains also served to 
indicate greater than^xpetted growth for participairts in 
each grade and subject area, but particularly in^^the- 
matics. Though the gams lessen in each succeeding grade 
. level, they remain fis indicators 'of movement m tftfe right 
direction across all of the grades' served by the'<:y3nsoh- * 
dated fundmg program. , * ^ * 

In 1976-77 readinj scores showed that approximately 
25 percent of the pupils gained' two months or more for 
every month of instructiort. In fact, 8 percent of {he * 
* pupjls gained'-thcee months o^more for .each month of 
instruction. In mathematics, percent gained two 
jnbnths or more for each month-of instructipn, apd 15* 
percent gained three months or mwe-. ' ' 

While most Vf* the specific school ^'ectives ^for 
reading and n\aih werQ attained, there are a numbej of 
operatiorral problems that need resolution. In interviews 
conducted at midyear, teachers \and 'aides in many 
schools expressed cQntinuing^ concern over the^record- 
'"Reeping loads require^-^^r participation ih specfal 
^ pro^ams^ particular!/^ :h^se relating to individualizing 
instruction. In soi((e schools profiles showig student 
" progress in mastering skills were not being kept, or were 
. not up 'to 'date. In some schools there was\some 
con fusion onuhe part of teacfiers as to the valine and 
necessity of keeping such detailed rec6rds.''Evaluators 
reported" these and other findings tq cdmpensalory 
^ education personnel at that time; and^ to tKe extent 
possible, they followed up to see wha^ Kelp could be 
given to Jhe schools. There is still a need .for further 
attention, especially since ^read^pg and nwth ar* con- . , 
§idered high^pnonty district and school comfJoijirtits. 

Classroom ^rnanagement ^problems iiTcrease as pfo-y/ 
grams become more individualized. In .some cases 
because' of increased paperwork, recordkeeping, and the 
time needed to Inanage other adults, the result may be 
less time available for^working with individual ^^t«dents. 

Jeachers arid aides' reported continuing inservice 
trailing *nee4s in the areas' of feadyig and mathematics, 
bilingu^ and multicultural education, ^d individualiza- 
tion and classroom management. There is still s(5me need ^ 
I for cl^^ren short-range and long-range goals and objec- * 
tives across the e/i tire compensat.ory ^education program 
arid for the setting of pnonties at the'school andiclistnct 
.levels. ' ^ 
" » ' * 

Program personnel .and evalua^on persoi/nci need to 
develop cooperatively clearer definitions of indivi- , 
dualized instruction to ensure more effective evaluation - 
in the future. * , ^ * 

• ' Over" three-quarters oT the teacher^Jo{ind the follow- ;^ 
mg three activities to be^the most effective m*staff ' . 

' development: inservice workshops, individual assistance 
froip -specialists, and classroom visitations: Classroo^i " ■ 

* demonstrations were rated by 53 percent of the*t6acheh 
. as being' the least effective strategy in staff development. 



There has been 'a relative decline ,in gains at grade 
three. . - , • 

Bilingual-bici^tural educatioa objectives were met in 
7 of 1.3 instances. In cases of nonatt|inment. dafa 
f were not availaljle for analysis. , t 

I referred three children and got no results or 
conference's with ^he psychologist about any^of Chem. 

¥ The topics for staff development coiild be more 
relevjynt to the actual needs of particular schools and 
type.s of students an'd designed on the basis of the 

• experience of t^e staff involved. 

Longitudinal Studies 

, Trends . . . include an almost steady climb in pojt-test 

grade equivalent scores since 19^0 m grades four 

' througji SIX but a vaned patte/n in grade three. Sizable 

gains over previous years are shown . . . m grades three, 

five, arid six. ' • . 
» ■ 

In matijpmaUcs, at the end of the third full ECE year., 
graces one and two' were making better than month-for- 
month gains, but grade three was falling behind. monih- 
for-nionth progress.^ " ^ * ' 

" The number of pupils scormg above 'the 50th percen- 
- tile (Q2) has increased morelhan 1 i perceftt-smcfi May, 
• 197T. The number of pupils sconng above the 25th 
per^ntile (Qj) has increased by •9-percent since May, 

. . . • % . 

Across the entire consolidated pjjogram^ however, the 
^ number c3f parents said to be mvolved rsincreasiiig each 
year. For example, while 550 'parents were listed as 
regular participants in 1974-75; th^ figure for 1975-76 
' exceeded 1,300. 

, ' ' In almost every school the pattern of growth, 
n/ieasured ^nually from fall to spnng, shows what is 
sometimes called the *'sawtooth" effect -substantial 
. growth during the school year but a loss dunng the 
summer. The siimmer idss appears 't(f be more pro- 
nounced among^low-scoring -pupils and is typically 
greater in njathematics than^n-feadihg.' 

The most encouraging sign. oF^air has^ ^een. the 
ffteadinejs ofthe improvement made by pupih over the 
^ years of the Title I program. It appears that success ful 
progr_ams require time for training the staff in how \o 
imple;^ent the programs and. that staff comrrtitment,to 
improved instructional programs has a cumulative effect 
, leading to continueid growth. * ' . . 

We have the Comprehensive Tests of Basic Skills 
reading and mathematics scores for local chdiren who 

* were tested' in October, r974, in grade four and who 
were still attepdmg the same ESEA Title I schools in 
May, 1977, whenMhey we^e 'post-tested m grade six. 
^While the analysis of the gams^made dunng this penod 
of 37 school months may^ not prove the yalue of the 
ESE/fTnie I program, the longitudinal data fcJr these 
children certainly weaken the cntiasm that **compensa- 



tory education programs show only temporary gains" so ^ ^ Generai Concerns 
far as ihe local program is concerned. 



A careful f(3Ilow-up was conducted tg/md out what 
had happened to those pupils w^ose scores on the CTBS 
' tests m October, 1974, placed Ahem m the bottom 
5 quarter on the basis of national norms. . . > A very high 
proportion of these weajcest pupils m gra*de fouix^tit 
found to have moved "out of the^bottorp/quartet when 

eated oa^he basis of grade six horrffi m May? 1977 
Modifications ' ' \ 

Further clanfication of the roles and responsibibties 
of central office and ^ area oiTiQt administrative y^nd 
resource personnel as well as definitions t)f .services 
available to sch(;>ol sites from these offices is needed. ' 

The utilization of the reading* -specialist m the 
classroonji is desirable for limited trammg This trlaimg' 

• * should be followed b> an independent program of each 

Ueacher and specialist) with. half a class This encourages 
folloN^-up ofMnstrtsriijp b\ th^ specialist 

. A concentrated euort to improve reading comprehen- 
^ src>n skill/needs to be planned and, carried out at each : 
school site Some techniques that could prove helpful 
are \ * " 

• Small groups for reading instruction 

• ^Or^d reading^DN the teacher on a regularly sctl^d- 
uleo daily basis , » 

• Selection and pufchasa^ of matenals designed to " 
^pro\Tde pracuce m comprehension 5,kiHs 

• Timelv daily feedback 'to 'students on their etTorts 
^ c m this area ' \ 

'Analyze carefully the potential role of manipulative 
jnaferials in each school Sre's rhathemafics program and 
the relauoi^ship of such matenals to the chosen basal 
t^xtbogks « * ^ 

Future 'distn*>t ECE planning should preserve t^e 
positive aspect's of parent par;ictpa'tion. ase-of aides, and 
restructuring efforts., * ' • / ' 

Practices that may tend to loN^er pupil tim'e spent on 

* basic, skills learning under directed instruction should be 
examined * , . ' ^ 

Staff development objectives should be mea;surable 
These objectives must be based on problems revealed by 
the evaluation tTndings with regard to 

, * Maintenance of direct instruct^ for pupils 

• Effective management of aides 

• Effective use of volunteers * 

• *Systerms for efficient j^ecordkeeping and inserMce 
. training for better mAiaeement of individualized 

» instruction . , ^, 

.Aux/liary services personnel should analyze (he causfes 
for the widely varying ratings for their activities to see 
^^hether. with the available resources, ihe delivery of 
•services can be improved . , 

More Filipino culture books and materials should be 
added as parents'have requ^ted^ 



Available data suggest that new pupils are likely to 
have lower reading and mathematics scbres than pupils 
who have mov^d away. 

A need exists for local evaluataon X)f specially funded 
programs The authors are convmced .that evaluation 
designs that are m&re appropriate than those that can be 
conducted withm the stale -re qui red evaluation frame- 
work must .be developed. For instance, the nlagnitude i 
and nature of the Overall recordkeepmg load inherent in 
the conduct of special progrims* needs to be'^assessed ' 
• The impact recordkeeping on the quabty .of instruc- . 
tion also needs to h/e^ assessed. Wuhin the state's 
evaluation framework Arfd design, neither the mandate 
nor the resources exist to 'studV such problems in 
addition, evaluation >9pecialists should help administra- 
tors i^entify^ specific * strate^es Schooj staffs Thave 
develof>ed ihat enable them \o vope with requixements 
v^hile buii'ding and maintainrng high quahiy instAictional 
programs in this ^^ay school staffs having pro'olems can 
receive more effects e and more mdividuaiize^ adminis- 
trative support 

Interpretation of Findings 

Too often a wnter seems to cry "failure" pnmanly 
because a program has./iot attained those goals that the 
, ^*nter has selected as the true%dicators of success The 
concepV of a prograp bemg designed to meet cer,t^ ' 
well-defined goals or objectWes that are based on 
documented needs and that are bemg evaluated, m furn. 
m relation to those goals and objectives seems to be a 
w*ell-accepted concept Nevertheless, it- appears to bfc a 
concept that is too often^cast^aside when wntmg for a 
national audience The cause of "fmding but" so v^e can 
••follow.^ up^ would^usuailv be better ser%*e<i if* the cntic 
would determine whether his or -her criticism should be 
directecf to\^ard the goals and objectives sei|cted for the 
program rather than tovvard those results that he or she 
interprets as "failure **i « ^ 

Having S-i perpent of the objectives - typically 
exceeded, completely attained. -^r suostantjafly attain^ 

. (Joes npt sound like failure Wliether that figure repre- J 
sents adequate or very satisYactory achievement can 

.-^obably best be answered by the school 'staffs, v^ho 
kno\\ what bes behind the numbers ^ 

in a normal population sampfe. however. 50 percent 
of the partioipantiare.-by deilnition. betov^ th6 median. 
Thus, while the program objective is. eminently com- 
mendable and .measurable, it may be unreabstic, it is 
mp|p reabstic to expect jhat only the excessive percent 
. of^tudenu below the national median be consistently - . 
diminished or ehminated ' ' / 

Before going on to the body of this report, a briei' ' 
^ discussion of academic growth and several othex items 
' relative to the analysis must be considered Bnelly when. - 
exammmg the .test analysis 'tables.' grov^th can be 
cortsidered "greater than e^tpected" ifrcqraparison to the 
national norming populatiori if 'the me^n percentile 



score of 'a riiaiched sample of studen/s increases from^ 
pr^l^ to post -test " ' L ^ ^ 

. With niimerous\riables opera^ye m the mfluenvlng 
of teadiii^ scores, it is difficult to jpw^n) conclusions 
on the^asis of one or tvro matched|groiips. 

• • • 

'To assess growth m ^eadin^ and ijailTeihaiics.skallsr, 
educators 'typically looJc at a* group's ajerlrje" grade 
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equivalent scores at two points in time to see whether, 
tlie group IS scoring more like the national sample, or . 
"norm** ^oup. The dififcrence in grade equivalent scores 
on t^o different testings is usually ufwpreted as being- 
"growth** in months or years. The publishers use 
sophisticated projectioac^.' and estunaies to^ arrives at the 
grade and month estimates for most scores, but they are 
still onl> estimates G-rowth interpretations sltould be ^ 
made with caution. V' 
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Other Publications Available 
from the Depa(^nient df Education 

^ Evaluation Report bf\fultiple-^unded Programs, 1^76-77, is one of appro;jumately ^0 publicaUons . 
which are availablefrom the California Slate Department of Eciucation. Some offlTe more recent 

• publications or thosemoa widely use<f are the following: " * . 

AdminiitjBUCn of the SchooJ Dbirici RisrManagemeni Progrim (1977) $ 2iO 

An Assessment of the Writing Perfonnanc« of California High School Seniors (1977) 2.75 
Axtertiancc and EmoUment Accounting and Reporting (1977) \^ , v 2.*80 

BibliographyoflnstxuctionalMaterulsfor th'eTcachlngofFr^ftch(1977), • - " [^q 

# 4 Bibliography of Instnictional Materials fot4he Teaching of Portuguese (1976) ' ' .75 * 

Bicyde Rules of the RcXadin)CaUfornia( 1977) ' - ' ' ' ^ 1^0 

C^orma Guftie to Traffic Safety Education ( 1 976) .3^6? 

California' Pnvaie School Directory 1977 ♦ • . { ' 5 00 

Cahforma School D^tory 1978 ^ ' . I' 11 00 '* 

y ' Cabfomia^hool Accourifi^ Manual ( 1 976^^^^'. mcfiSikg 1 9^78 revui^nif) . ^ 1 65 * 

California Sch(>ol Effectiveness Study (1977)^^^ ' * ^5, 

C^ifornia School Lighting Design and EvaluatK>nN^78) * ' ^ ' Ks 

California Teachers Salanes and Salary Schedules, 1977.78 (1978) * 10 00 

•Discussion Guide for the California School Improvement Program ( t978) i * 1 

Distnct Paid Insurance ^ogrannm. California School' Distrfas, 1977-78 (19t8) . 2^0 

EnghshLanguag<^ramftu'ork for CaUfomia Public Schools (1976) . ' » , 150 < 

• Establishing School Site Councils: The California School Impro>-emtnt Program ( 1977) ' , , ' 150 ' 

Guidefor Multicultural Education: Content and Context (1977)' - ' > •'1.25."' 

Guid* for Ongoii^>Planning( 1977) * , * • .0 - ' i,\Q 

Handbook for Reporting and Using Test Results (1976) , » ^ r ' 8il) 

X.HandbookR^^ing the Pnv*ac/ and DiKlosure of Pupil Records (1978) ' • - * ^ ^ .65 

Health Instrucuon Framr*'oric for California Pubbc^ooIs( 1^7 81^ iit * ' * ^ .135 

Hospitality OccapauonsCumcxilunvCuidc CI 97 7) 7^ X ^L^^^j ^ ' 3 00 

'PhysacalEducation'forChiWrenv Age$'FoutThroug#Nine(1^78)' / ^ ''^iO 

Planning Handbook (1978\ ^ ' * * V • ^» ^ ^50"^ 

' Site Management (1977) < • ' 15o' 

Social Sdcnces ExiucaDon Framework for California Public Schools (1975) ' * 110, 

^ Students* Rights and Responsibilities Handbook ( 1978) ' ^ - 150 

Orders shjould be dircQted to: " ^ * 

, ^ . California State Department of Educauon ^ ^ 

''^ . P.O. Box 271 ^ , ^ ' ' *' : 

Sacramento. C A 95802 * ' . p ' ,\ 

Remittance Df pur.cliase ord^r must accompany order. Purchase orders without checks are acc^led onl/ 
from government agencies ,in California, Six percent sales tax should be added to all orders from Califemia 
purchasers.* ^ ' ^ ^ v^ l ^ 

A compleifijist of pubhcatiSns available frfoifi'the Department maybe obtained by^wTitmg to the address 
listed above. " ' 
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